Collaborative artificial intelligence method and system

ABSTRACT

A method and system of audibly broadcasting responses to a user based on user queries about a specific patient report, the method comprising receiving an audible query from the user to a microphone coupled to a collaboration device, identifying at least one intent associated with the audible query, identifying at least one data operation associated with the at least one intent, associating each of the at least one data operations with a first set of data presented on the report, executing each of the at least one data operations on a second set of data to generate response data, generating an audible response file associated with the response data and providing the audible response file for broadcasting via a speaker coupled to the collaboration device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Pat. Application 16/852,194,filed Apr. 17, 2020, which is titled “COLLABORATIVE ARTIFICIALINTELLIGENCE METHOD AND SYSTEM” which claims priority to U.S.Provisional Pat. Application No. 62/871,667 which is titled“COLLABORATIVE ARTIFICIAL INTELLIGENCE METHOD AND SYSTEM”, which wasfiled Jul. 8, 2019, U.S. Provisional Pat. Application No. 62/855,646which is titled “COLLABORATIVE ARTIFICIAL INTELLIGENCE METHOD ANDAPPARATUS” which was filed on Jun. 24, 2019, and to U.S. ProvisionalPat. Application No. 62/835,339 which is titled “COLLABORATIVEARTIFICIAL INTELLIGENCE METHOD AND APPARATUS” which was filed on Apr.17, 2019. Each application listed is incorporated herein by reference inits entirety.

APPLICATIONS INCORPORATED BY REFERENCE

Each of the following U.S. Pat. Applications is incorporated herein inits entirety by reference.

-   (1) U.S. Pat. Application No. 16/657,804 which is titled “DATA BASED    CANCER RESEARCH AND TREAMENT SYSTEMS AND METHODS,” which was filed    on Oct. 18, 2019;-   (2) U.S. Pat. Application No. 16/671,165 which is titled “USER    INTERFACE, SYSTEM, AND METHOD FOR COHORT ANALYSIS,” which was filed    on Dec. 31, 2019;-   (3) U.S. Pat. Application No. 16/732,168 which is titled “A METHOD    AND PROCESS FOR PREDICTING AND ANALYZING PATIENT COHORT RESPONSE,    PROGRESSION, AND SURVIVAL,” which was filed on Dec. 31, 2019.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not applicable.

BACKGROUND OF THE DISCLOSURE

The field of this disclosure is systems for accessing and manipulatinglarge complex data sets in ways that enable system users to develop newinsights and conclusions with minimal user-interface friction hinderingaccess and manipulation.

The present disclosure describes innovations that will be described inthe context of an exemplary healthcare worker that collaborates withpatients to diagnose ailment states, prescribe treatments, andadminister those treatments to improve overall patient health. Inaddition, while many different types of healthcare workers (e.g.,doctors, psychologists, physical therapists, nurses, administrators,researchers, insurance experts, pharmacists, etc.) in many differentmedical disciplines (e.g., cancer, Alzheimer’s disease, Parkinson’sdisease, mental illnesses, cardiology, immunology, infectious disease,and diabetes) will benefit from the disclosed innovations, unlessindicated otherwise, the innovations will be described in the context ofan exemplary oncologist/researcher (hereinafter “oncologist”) whocollaborates with patients to diagnose cancer states (e.g., allphysiological, habit, history, genetic and treatment efficacy factors),understand and evaluate existing data and guidelines for patientssimilar to their patient, prescribe treatments, administer thosetreatments, and observes patient outcomes, all to improve overallpatient health, and/or who performs medical research in cancer.

Many professions require complex thought where people need to considermany factors when selecting solutions to encountered situations,hypothesize new factors and solutions and test new factors and solutionsto make sure that they are effective. For instance, oncologistsconsidering specific patient cancer states, optimally should considermany different factors when assessing the patient’s cancer state as wellas many factors when crafting and administering an optimized treatmentplan. For example, these factors include the patient’s family history,past medical conditions, current diagnosis, genomic/molecular profile ofthe patient’s hereditary DNA and of the patient’s tumor’s DNA, currentnationally recognized guidelines for standards of care within thatcancer subtype, recently published research relating to that patient’scondition, available clinical trials pertaining to that patient,available medications and other therapeutic interventions that may be agood option for the patient and data from similar patients. In addition,cancer and cancer treatment research are evolving rapidly so thatresearchers need to continually utilize data, new research and newtreatment guidelines to think critically about new factors andtreatments when diagnosing cancer states and optimized treatment plans.

In particular, it is no longer possible for an oncologist to be familiarwith all new research in the field of cancer care. Similarly, it isextremely challenging for an oncologist to be able to manually analyzethe medical records and outcomes of thousands or millions of cancerpatients each time an oncologist wants to make a specific treatmentrecommendation regarding a particular patient being treated by thatoncologist. As an initial matter, oncologists often do not even haveaccess to health information from institutions other than their own. Inthe United States, implementation of the federal law known as the HealthInsurance Portability and Accountability Act of 1996 (“HIPAA”) placessignificant restrictions on the ability of one health care provider toaccess health records of another health care provider. In addition,health care systems face administrative, technical, and financialchallenges in making their data available to a third party foraggregation with similar data from other health care systems. To theextent health care information from multiple patients seen at multipleproviders has been aggregated into a single repository, there is a needfor a system and method that structures that information using a commondata dictionary or library of data dictionaries. Where multipleinstitutions are responsible for the development of a single, aggregatedrepository, there can be significant disagreement over the structure ofthe data dictionary or data dictionaries, the methods of accessing thedata, the individuals or other providers permitted to access the data,the quantity of data available for access, and so forth. Moreover, thescope of the data that is available to be searched is overwhelming forany oncologist wishing to conduct a manual review. Every patient hashealth information that includes hundreds or thousands of data elements.When including sequencing information in the health information to beaccessed and analyzed, such as from next-generation sequencing, thevolume of health information that could be analyzed grows intensely. Asingle FASTQ or BAM file that is produced in the course of whole-exomesequencing, for instance, takes up gigabytes of storage, even though itincludes sequencing for only the patient’s exome, which is thought to beabout 1-2% of the whole human genome.

In this regard, an oncologist may have a simple question - “what is thebest medication for this particular patient?” - the answer to whichrequires an immense amount of health information, analytical softwaremodules for analyzing that information, and a hardware framework thatpermits those modules to be executed in order to provide an answer.Almost all queries/ideas/concepts are works in progress that evolve overtime as critical thinking is applied and additional related factors andfactor relationships are recognized and/or better understood. Allqueries start as a hypothesis rooted in consideration of a set ofinterrelated raw material (e.g., data). The hypothesis is usually testedby asking questions related to the hypothesis and determining if thehypothesis is consistent and persists when considered in light of theraw material and answers to the questions. Consistent/persistenthypothesis become relied upon ideas (i.e., facts) and additional rawmaterial for generating next iterations of the initial ideas as well ascompletely new ideas.

When considering a specific cancer state, an oncologist considers knownfactors (e.g., patient conditions, prior treatments, treatment efficacy,etc.), forms a hypothesis regarding optimized treatment, considers thathypothesis in light of prior data and prior research relating similarcancer states to treatment efficacies and, where the prior dataindicates high efficacy regarding the treatment hypothesis, mayprescribe the hypothesized treatment for a patient. Where data indicatespoor treatment efficacy the oncologist reconsiders and generates adifferent hypothesis and continues the iterative testing and conclusioncycle until an efficacious treatment plan is identified. Cancerresearchers perform similar iterative hypothesis, data testing andconclusion processes to derive new cancer research insights.

Tools have been and continue to be developed to help oncologistsdiagnose cancer states, select and administer optimized treatments andexplore and consider new cancer state factors, new cancer states (e.g.,diagnosis), new treatment factors, new treatments and new efficacyfactors. For instance, massive cancer databases have been developed andare maintained for access and manipulation by oncologists to explorediagnosis and treatment options as well as new insights and treatmenthypothesis. Computers enable access to and manipulation of cancer dataand derivatives thereof.

Cancer data tends to be voluminous and multifaceted so that many usefulrepresentations include substantial quantities of detail and specificarrangements of data or data derivatives that are optimally visuallyrepresented. For this reason, oncological and research computerworkstations typically include conventional interface devices like oneor more large flat panel display screens for presenting datarepresentations and a keyboard, mouse, or other mechanical input devicefor entering information, manipulating interface tools and presentingmany different data representations. In many cases a workstationcomputer/processor runs electronic medical records (EMR) or medicalresearch application programs (hereinafter “research applications”) thatpresent different data representations along with on screen cursorselectable control icons for selecting different data access andmanipulation options.

While conventional computers and workstations operate well as dataaccess and manipulation interfaces, they have several shortcomings.First, using a computer interface often requires an oncologist to clickmany times, on different interfaces, to find a specific piece ofinformation. This is a cumbersome and time consuming process which oftendoes not result in the oncologist achieving the desired result andreceiving the answer to the question they are trying to ask.

Second, in many cases it is hard to capture hypothetical queries whenthey occur and the ideas are not followed up on in a timely fashion orare lost forever. Queries are not restricted to any specific timeschedule and therefore often occur at inconvenient times when anoncologist is not logged into a workstation and using a researchapplication usable to capture and test the idea. For instance, anoncologist may be at home when she becomes curious about some aspect ofa patient’s cancer state or some statistic related to one of herpatients or when she first formulates a treatment hypothesis for aspecific patient’s cancer state. In this case, where the oncologist’sworkstation is at a remote medical facility, the oncologist cannoteasily query a database or capture or test the hypothesis.

Also, in this case, even if the oncologist can use a laptop or otherhome computer to access a research application from home, the frictioninvolved with engaging the application often has an impeding effect. Inthis regard, application access may require the oncologist to retrieve alaptop or physically travel to a stationary computer in her home, bootup the computer operating system, log onto the computer (e.g., enteruser name and password), select and start a research application,navigate through several application screenshots to a desired databaseaccess tool suite and then enter a query or hypothesis defininginformation in order to initiate hypothesis testing. This applicationaccess friction is sufficient in many cases to dissuade immediatequeries or hypothesis capture and testing, especially in cases where anoncologist simply assumes she will remember the query or hypothesis thenext time she access her computer interface. As anyone who has a lot ofideas knows, ideas are fleeting and therefore ideas not immediatelycaptured are often lost. More importantly, oncologists typically havelimited amounts of time to spend on each patient case and need to havetheir questions and queries resolved immediately while they areevaluating information specific to that patient.

Third, in many cases a new query or hypothesis will occur to anoncologist while engaged in some other activity unrelated to oncologicalactivities. Here, as with many people, immediate consideration andtesting via a conventional research application is simply notconsidered. Again, no immediate capture can lead to lost ideas.

Fourth, in many cases oncological and research data activities willinclude a sequence of consecutive questions or requests (hereinafter“requests”) that home in on increasingly detailed data responses wherethe oncologist/researcher has to repeatedly enter additional input todefine next level requests as intermediate results are not particularlyinteresting. In addition, while visual representations of data responsesto oncological and research requests are optimal in many cases, in othercases visual representations tend to hamper user friendliness and caneven be overwhelming. In these cases, while the visual representationsare usable, the representations can require appreciable time and effortto consume presented information (e.g., reading results, mentallysummarizing results, etc.). In short, conventional oncologicalinterfaces are often clunky to use.

Moreover, today, oncologists and other professionals have no simplemechanism for making queries of large, complex databases and receivinganswers in real time, without needing to interact with electronic healthrecord systems or other cumbersome software solutions. In particular,there is a need for systems and methods that allow a provider to query adevice using his or her voice, with questions relating to the optimalcare of his or her patient, where the answers to those questions aregenerated from unique data sets that provide context and new informationrelative to the patient, including vast amounts of real world historicalclinical information combined with other forms of medical data such asmolecular data from omics sequencing and imaging data, as well as dataderived from such data using analytics to determine which path is mostoptimal for that singular patient

Thus, what is needed is an intuitive interface for complex databasesthat enables oncologists, researchers, and other professionals anddatabase users to access and manipulate data in various ways to generatequeries and test hypothesis or new ideas thereby thinking through thoseideas in the context of different data sets with minimal access andmanipulation friction. It would be advantageous if the interface werepresent at all times or at least portable so that it is availableessentially all the time. It would also be advantageous if a systemassociated with the interface would memorialize user-interfaceinteractions thereby enabling an oncologist or researcher to reconsiderthe interactions at a subsequent time to re-engage for the purpose ofcontinuing a line of questions or hypothesis testing without losingprior thoughts.

It would also be advantageous to have a system that captures anoncologist’s thoughts for several purposes such as developing betterhealthcare aid systems, generating automated records and documents andoffering up services like appointment, test and procedure scheduling,prescription preparation, etc.

It would also be advantageous to have an interface available acrossseveral different form factors.

SUMMARY OF THE DISCLOSURE

It has been recognized that a relatively small and portable voiceactivated and audio responding interface device (hereinafter“collaboration device”) can be provided enabling oncologists to conductat least initial database access and manipulation activities. In atleast some embodiments, a collaboration device includes a processorlinked to each of a microphone, a speaker and a wireless transceiver(e.g., transmitter and receiver). The processor runs software forcapturing voice signals generated by an oncologist. An automated speechrecognition (ASR) system converts the voice signals to a text file whichis then processed by a natural language processor (NLP) or otherartificial intelligence module (e.g., a natural language understandingmodule) to generate a data operation (e.g., commands to perform somedata access or manipulation process such as a query, a filter, amemorialization, a clearing of prior queries and filter results, noteetc.).

In at least some embodiments the collaboration device is used within acollaboration system that includes a server that maintains andmanipulates an industry specific data repository. The data operation isreceived by the collaboration server and used to access and/ormanipulate data the database data thereby generating a data response. Inat least some cases, the data response is returned to the collaborationdevice as an audio file which is broadcast to the oncologist as a resultassociated with the original query.

In some cases the voice signal to text file transcription is performedby the collaboration device processor while in other cases the voicesignal is transmitted from the collaboration device to the collaborationserver and the collaboration server does the transcription to a textfile. In some cases the text file is converted to a data operation bythe collaboration device processor and in other cases that conversion isperformed by the collaboration server. In some cases the collaborationserver maintains or has access to the industry specific database so thatthe server operates as an intermediary between the collaboration deviceand the industry specific database.

In at least some embodiments the collaboration device is a dedicatedcollaboration device that is provided solely as an interface to thecollaboration server and industry specific database. In these cases, thecollaboration interface device may be on all the time and may only run asingle dedicated application program so that the device does not requireany boot up time and can be activated essentially immediately via asingle activation activity performed by an oncologist.

For instance, in some cases the collaboration device may have motionsensors (e.g., an accelerometer, a gyroscope, etc.) linked to theprocessor so that the simple act of picking up the device causes theprocessor to activate an application. In other cases the collaborationdevice processor may be programmed to “listen” for the phrase “Heyquery” and once received, activate to capture a next voice signalutterance that operates as seed data for generating the text file. Inother cases the processor may be programmed to listen for a differentactivation phrase, such as a brand name of the system or a combinationof a brand name plus a command indication. For instance, if the brandname of the system is “One” then the activation phrase may be “One” or“Go One” or the like. In still other cases the collaboration device maysimply listen for voice signal utterances that it can recognize asoncological queries and may then automatically use any recognized queryas seed data for text generation.

In addition to providing audio responses to data operations, in at leastsome cases the system automatically records and stores data operations(e.g., data defining the operations) and responses as a collaborationrecord for subsequent access. The collaboration record may include oneor the other or both of the original voice signal and broadcast responseor the text file and a text response corresponding to the data response.Here, the stored collaboration record provides details regarding theoncologist’s search and data operation activities that helpautomatically memorialize the hypothesis or idea the oncologist wasconsidering. In a case where an oncologist asks a series of queries,those queries and data responses may be stored as a single line ofquestioning so that they together provide more detail for characterizingthe oncologist’s initial hypothesis or idea. At a subsequent time, thesystem may enable the oncologist to access the memorialized queries anddata responses so that she can re-enter a flow state associatedtherewith and continue hypothesis testing and data manipulation using aworkstation type interface or other computer device that includes adisplay screen and perhaps audio devices like speakers, a microphone,etc., more suitable for presenting more complex data sets and datarepresentations.

In addition to simple data search queries, other voice signal dataoperation types are contemplated. For instance, the system may supportfilter operations where an oncologist voice signal message defines asub-set of the industry specific database set. For example, theoncologist may voice the message “Access all medical records for malepatients over 45 years of age that have had pancreatic cancer since1990”, causing the system to generate an associated subset of data thatmeet the specified criteria.

Importantly, some data responses to oncological queries will be “audiosuitable” meaning that the response can be well understood andcomprehended when broadcast as an audio message. In other cases a dataresponse simply may not be well suited to be presented as an audiooutput. For instance, where a query includes the phrase “Who is thepatient that I saw during my last office visit last Thursday?”, an audiosuitable response may be “Mary Brown.” On the other hand, if a query is“List all the medications that have been prescribed for males over 45years of age that have had pancreatic cancer since 1978” and theresponse includes a list of 225 medications, the list would not be audiosuitable as it would take a long time to broadcast each list entry andcomprehension of all list entries would be dubious at best.

In cases where a data response is optimally visually presented, thesystem may take alternate or additional steps to provide the response inan intelligible format to the user. The system may simply indicate aspart of an audio response that response data would be more suitablypresented in visual format and then present the audio response. If thereis a proximate large display screen, such as a computer monitor or atelevision (TV) such as a smart TV, the system may pair with thatdisplay and present visual data with or without audio data. The systemmay simply indicate that no suitable audio response is available. Insome embodiments, the system may pair with a computational device thatincludes a display, such as a smartphone, tablet computer, etc.

Thus, at least some inventive embodiments enable intuitive and rapidaccess to complex data sets essentially anywhere within a wirelesscommunication zone so that an oncologist can initiate thought processesin real time when they occur. By answering questions when they occur,the system enables oncologists to dig deeper in the moment into data andcontinue the thought process through a progression of queries. Someembodiments memorialize an oncologist’s queries and responses so that atsubsequent times the oncologist can re-access that information andcontinue queries related thereto. In cases where visual and audioresponses are available, the system may adapt to provide visualresponses when visual capabilities are present or may simply store thevisual responses as part of a collaboration record for subsequent accesswhen an oncologist has access to a workstation or the like.

In at least some embodiments the disclosure includes a method forinteracting with a database to access data therein, the method for usewith a collaboration device including a speaker, a microphone and aprocessor, the method comprising the steps of associating separate setsof state-specific intents and supporting information with differentclinical report types, the supporting information including at least oneintent-specific data operation for each state-specific intent, receivinga voice query via the microphone seeking information, identifying aspecific patient associated with the query, identifying a state-specificclinical report associated with the identified patient, attempting toselect one of the state-specific intents associated with the identifiedstate-specific clinical report as a match for the query, upon selectionof one of the state-specific intents, performing the at least one dataoperation associated with the selected state-specific intent to generatea result, using the result to form a query response and broadcasting thequery response via the speaker.

In some cases the method is for use with at least a first database thatincludes information in addition the clinical reports, the methodfurther including, in response to the query, obtaining at least a subsetof the information in addition to the clinical reports, the step ofusing the result to form a query response including using the result andthe additional obtained information to form the query response.

In some cases the at least one data operation includes at least one dataoperation for accessing additional information from the database, thestep of obtaining at least a subset includes obtaining data per the atleast one data operation for accessing additional information from thedatabase.

Some embodiments include a method for interacting with a database toaccess data therein, the method for use with a collaboration deviceincluding a speaker, a microphone and a processor, the method comprisingthe steps of associating separate sets of state-specific intents andsupporting information with different clinical report types, thesupporting information including at least one intent-specific primarydata operation for each state-specific intent, receiving a voice queryvia the microphone seeking information, identifying a specific patientassociated with the query, identifying a state-specific clinical reportassociated with the identified patient, attempting to select one of thestate-specific intents associated with the identified state-specificclinical report as a match for the query, upon selection of one of thestate-specific intents, performing the primary data operation associatedwith the selected state-specific intent to generate a result, performinga supplemental data operation on data from a database that includes datain addition to the clinical report data to generate additionalinformation, using the result and the additional information to form aquery response and broadcasting the query response via the speaker.

Some embodiments include a method of audibly broadcasting responses to auser based on user queries about a specific patient molecular report,the method comprising receiving an audible query from the user to amicrophone coupled to a collaboration device, identifying at least oneintent associated with the audible query, identifying at least one dataoperation associated with the at least one intent, associating each ofthe at least one data operations with a first set of data presented onthe molecular report, executing each of the at least one data operationson a second set of data to generate response data, generating an audibleresponse file associated with the response data and providing theaudible response file for broadcasting via a speaker coupled to thecollaboration device.

In at least some cases the audible query includes a question about anucleotide profile associated with the patient. In at least some casesthe nucleotide profile associated with the patient is a profile of thepatient’s cancer. In at least some cases the nucleotide profileassociated with the patient is a profile of the patient’s germline. Inat least some cases the nucleotide profile is a DNA profile. In at leastsome cases the nucleotide profile is an RNA expression profile. In atleast some cases the nucleotide profile is a mutation biomarker.

In at least some cases the mutation biomarker is a BRCA biomarker. In atleast some cases the audible query includes a question about a therapy.In at least some cases the audible query includes a question about agene. In at least some cases the audible query includes a question abouta clinical data. In at least some cases the audible query includes aquestion about a next-generation sequencing panel. In at least somecases the audible query includes a question about a biomarker.

In at least some cases the audible query includes a question about animmune biomarker. In at least some cases the audible query includes aquestion about an antibody-based test. In at least some cases theaudible query includes a question about a clinical trial. In at leastsome cases the audible query includes a question about an organoidassay. In at least some cases the audible query includes a questionabout a pathology image. In at least some cases the audible queryincludes a question about a disease type. In at least some cases the atleast one intent is an intent related to a biomarker. In at least somecases the biomarker is a BRCA biomarker. In at least some cases the atleast one intent is an intent related to a clinical condition. In atleast some cases the at least one intent is an intent related to aclinical trial.

In at least some cases the at least one intent is related to a drug. Inat least some cases the drug intent is related to a drug ischemotherapy. In at least some cases the drug intent is an intentrelated to a PARP inhibitor intent. In at least some cases the at leastone intent is related to a gene. In at least some cases the at least oneintent is related to immunology. In at least some cases the at least oneintent is related to a knowledge database. In at least some cases the atleast one intent is related to testing methods. In at least some casesthe at least one intent is related to a gene panel. In at least somecases the at least one intent is related to a report. In at least somecases the at least one intent is related to an organoid process. In atleast some cases the at least one intent is related to imaging.

In at least some cases the at least one intent is related to a pathogen.In at least some cases the at least one intent is related to a vaccine.In at least some cases the at least one data operation includes anoperation to identify at least one treatment option. In at least somecases the at least one data operation includes an operation to identifyknowledge about a therapy. In at least some cases the at least one dataoperation includes an operation to identify knowledge related to atleast one drug (e.g., “What drugs are associated with high CD40expression?”). In at least some cases the at least one data operationincludes an operation to identify knowledge related to mutation testing(e.g., “Was Dwayne Holder’s sample tested for a KMT2D mutation?”). In atleast some cases the at least one data operation includes an operationto identify knowledge related to mutation presence (e.g., “Does DwayneHolder have a KMT2C mutation?”). In at least some cases the at least onedata operation includes an operation to identify knowledge related totumor characterization (e.g. “Could Dwayne Holder’s tumor be a BRCA2driven tumor?”). In at least some cases the at least one data operationincludes an operation to identify knowledge related to testingrequirements (e.g., “What tumor percentage does TEMPUS require for TMBresults?”). In at least some cases the at least one data operationincludes an operation to query for definition information (e.g., “Whatis PDL1 expression?”). In at least some cases the at least one dataoperation includes an operation to query for expert information (e.g.,“What is the clinical relevance of PDL1 expression?”; “What are thecommon risks associated with the Whipple procedure?”). In at least somecases the at least one data operation includes an operation to identifyinformation related to recommended therapy (e.g., “Dwayne Holder is inthe 88th percentile of PDL1 expression, is he a candidate forimmunotherapy?”). In at least some cases the at least one data operationincludes an operation to query for information relating to a patient(e.g., Dwayne Holder). In at least some cases the at least one dataoperation includes an operation to query for information relating topatients with one or more clinical characteristics similar to thepatient (e.g., “What are the most common adverse events for patientssimilar to Dwayne Holder?”).

In at least some cases the at least one data operation includes anoperation to query for information relating to patient cohorts (e.g.,“What are the most common adverse events for pancreatic cancerpatients?”). In at least some cases the at least one data operationincludes an operation to query for information relating to clinicaltrials (e.g., “Which clinical trials is Dwayne the best match for?”).

In at least some cases the at least one data operation includes anoperation to query about a characteristic relating to a genomicmutation. In at least some cases the characteristic is loss ofheterozygosity. In at least some cases the characteristic reflects thesource of the mutation. In at least some cases the source is germline.In at least some cases the source is somatic. In at least some cases thecharacteristic includes whether the mutation is a tumor driver. In atleast some cases the first set of data comprises a patient name.

In at least some cases the first set of data comprises a patient age. Inat least some cases the first set of data comprises a next-generationsequencing panel. In at least some cases the first set of data comprisesa genomic variant. In at least some cases the first set of datacomprises a somatic genomic variant. In at least some cases the firstset of data comprises a germline genomic variant. In at least some casesthe first set of data comprises a clinically actionable genomic variant.In at least some cases the first set of data comprises a loss offunction variant. In at least some cases the first set of data comprisesa gain of function variant.

In at least some cases the first set of data comprises an immunologymarker. In at least some cases the first set of data comprises a tumormutational burden. In at least some cases the first set of datacomprises a microsatellite instability status. In at least some casesthe first set of data comprises a diagnosis. In at least some cases thefirst set of data comprises a therapy. In at least some cases the firstset of data comprises a therapy approved by the U.S. Food and DrugAdministration. In at least some cases the first set of data comprises adrug therapy. In at least some cases the first set of data comprises aradiation therapy. In at least some cases the first set of datacomprises a chemotherapy. In at least some cases the first set of datacomprises a cancer vaccine therapy. In at least some cases the first setof data comprises an oncolytic virus therapy.

In at least some cases the first set of data comprises an immunotherapy.In at least some cases the first set of data comprises a pembrolizumabtherapy. In at least some cases the first set of data comprises a CAR-Ttherapy. In at least some cases the first set of data comprises a protontherapy. In at least some cases the first set of data comprises anultrasound therapy. In at least some cases the first set of datacomprises a surgery. In at least some cases the first set of datacomprises a hormone therapy. In at least some cases the first set ofdata comprises an off-label therapy. In at least some cases, the firstset of data comprises a gene editing therapy. In at least some cases,the gene editing therapy can be clustered regularly interspaced shortpalindromic repeats (CRISPR) therapy.

In at least some cases the first set of data comprises an on-labeltherapy. In at least some cases the first set of data comprises a bonemarrow transplant event. In at least some cases the first set of datacomprises a cryoablation event. In at least some cases the first set ofdata comprises a radiofrequency ablation. In at least some cases thefirst set of data comprises a monoclonal antibody therapy. In at leastsome cases the first set of data comprises an angiogenesis inhibitor. Inat least some cases the first set of data comprises a PARP inhibitor.

In at least some cases the first set of data comprises a targetedtherapy. In at least some cases the first set of data comprises anindication of use. In at least some cases the first set of datacomprises a clinical trial. In at least some cases the first set of datacomprises a distance to a location conducting a clinical trial. In atleast some cases the first set of data comprises a variant of unknownsignificance. In at least some cases the first set of data comprises amutation effect.

In at least some cases the first set of data comprises a variant allelefraction. In at least some cases the first set of data comprises a lowcoverage region. In at least some cases the first set of data comprisesa clinical history. In at least some cases the first set of datacomprises a biopsy result. In at least some cases the first set of datacomprises an imaging result. In at least some cases the first set ofdata comprises an MRI result.

In at least some cases the data comprises a CT result. In at least somecases the first set of data comprises a therapy prescription. In atleast some cases the first set of data comprises a therapyadministration. In at least some cases the first set of data comprises acancer subtype diagnosis. In at least some cases the first set of datacomprises an cancer subtype diagnosis by RNA class. In at least somecases the first set of data comprises a result of a therapy applied toan organoid grown from the patient’s cells. In at least some cases thefirst set of data comprises a tumor quality measure. In at least somecases the first set of data comprises a tumor quality measure selectedfrom at least one of the set of PD-L1, MMR, tumor infiltratinglymphocyte count, and tumor ploidy. In at least some cases the first setof data comprises a tumor quality measure derived from an image analysisof a pathology slide of the patient’s tumor. In at least some cases thefirst set of data comprises a signaling pathway associated with a tumorof the patient.

In at least some cases the signaling pathway is a HER pathway. In atleast some cases the signaling pathway is a MAPK pathway. In at leastsome cases the signaling pathway is a MDM2-TP53 pathway. In at leastsome cases the signaling pathway is a PI3K pathway. In at least somecases the signaling pathway is a mTOR pathway.

In at least some cases the at least one data operations includes anoperation to query for a treatment option, the first set of datacomprises a genomic variant, and the associating step comprisesadjusting the operation to query for the treatment option based on thegenomic variant. In at least some cases the at least one data operationsincludes an operation to query for a clinical history data, the firstset of data comprises a therapy, and the associating step comprisesadjusting the operation to query for the clinical history data elementbased on the therapy. In at least some cases the clinical history datais medication prescriptions, the therapy is pembrolizumab, and theassociating step comprises adjusting the operation to query for theprescription of pembrolizumab.

In at least some cases the second set of data comprises clinical healthinformation. In at least some cases the second set of data comprisesgenomic variant information. In at least some cases the second set ofdata comprises DNA sequencing information. In at least some cases thesecond set of data comprises RNA information. In at least some cases thesecond set of data comprises DNA sequencing information from short-readsequencing. In at least some cases the second set of data comprises DNAsequencing information from long-read sequencing. In at least some casesthe second set of data comprises RNA transcriptome information. In atleast some cases the second set of data comprises RNA full-transcriptomeinformation. In at least some cases the second set of data is stored ina single data repository. In at least some cases the second set of datais stored in a plurality of data repositories.

In at least some cases the second set of data comprises clinical healthinformation and genomic variant information. In at least some cases thesecond set of data comprises immunology marker information. In at leastsome cases the second set of data comprises microsatellite instabilityimmunology marker information. In at least some cases the second set ofdata comprises tumor mutational burden immunology marker information. Inat least some cases the second set of data comprises clinical healthinformation comprising one or more of demographic information,diagnostic information, assessment results, laboratory results,prescribed or administered therapies, and outcomes information.

In at least some cases the second set of data comprises demographicinformation comprising one or more of patient age, patient date ofbirth, gender, race, ethnicity, institution of care, comorbidities, andsmoking history. In at least some cases the second set of data comprisesdiagnosis information comprising one or more of tissue of origin, dateof initial diagnosis, histology, histology grade, metastatic diagnosis,date of metastatic diagnosis, site or sites of metastasis, and staginginformation. In at least some cases the second set of data comprisesstaging information comprising one or more of TNM, ISS, DSS, FAB, RAI,and Binet. In at least some cases the second set of data comprisesassessment information comprising one or more of performance status(including ECOG or Karnofsky status), performance status score, and dateof performance status.

In at least some cases the second set of data comprises laboratoryinformation comprising one or more of type of lab (e.g. CBS, CMP, PSA,CEA), lab results, lab units, date of lab service, date of molecularpathology test, assay type, assay result (e.g. positive, negative,equivocal, mutated, wild type), molecular pathology method (e.g. IHC,FISH, NGS), and molecular pathology provider. In at least some cases thesecond set of data comprises treatment information comprising one ormore of drug name, drug start date, drug end date, drug dosage, drugunits, drug number of cycles, surgical procedure type, date of surgicalprocedure, radiation site, radiation modality, radiation start date,radiation end date, radiation total dose delivered, and radiation totalfractions delivered.

In at least some cases the second set of data comprises outcomesinformation comprising one or more of Response to Therapy (e.g. CR, PR,SD, PD), RECIST score, Date of Outcome, date of observation, date ofprogression, date of recurrence, adverse event to therapy, adverse eventdate of presentation, adverse event grade, date of death, date of lastfollow-up, and disease status at last follow up. In at least some casesthe second set of data comprises information that has been de-identifiedin accordance with a de-identification method permitted by HIPAA.

In at least some cases the second set of data comprises information thathas been de-identified in accordance with a safe harborde-identification method permitted by HIPAA. In at least some cases thesecond set of data comprises information that has been de-identified inaccordance with a statistical de-identification method permitted byHIPAA. In at least some cases the second set of data comprises clinicalhealth information of patients diagnosed with a cancer condition.

In at least some cases the second set of data comprises clinical healthinformation of patients diagnosed with a cardiovascular condition. In atleast some cases the second set of data comprises clinical healthinformation of patients diagnosed with a diabetes condition. In at leastsome cases the second set of data comprises clinical health informationof patients diagnosed with an autoimmune condition. In at least somecases the second set of data comprises clinical health information ofpatients diagnosed with a lupus condition.

In at least some cases the second set of data comprises clinical healthinformation of patients diagnosed with a psoriasis condition. In atleast some cases the second set of data comprises clinical healthinformation of patients diagnosed with a depression condition. In atleast some cases the second set of data comprises clinical healthinformation of patients diagnosed with a rare disease.

In at least some embodiments, a method of audibly broadcasting responsesto a user based on user queries about a specific patient’s molecularreport is provided by the disclosure. The method can be used with acollaboration device that includes a processor and a microphone and aspeaker linked to the processor. The method can include storingmolecular reports for a plurality of patients in a system database,receiving an audible query from the user via the microphone, identifyingat least one intent associated with the audible query, identifying atleast one data operation associated with the at least one intent,accessing the specific patient’s molecular report, executing at leastone of the identified at least one data operations on a first set ofdata included in the specific patient’s molecular report to generate afirst set of response data, using the first set of response data togenerate an audible response file, and broadcasting the audible responsefile via the speaker.

In at least some cases the method can further include identifyingqualifying parameters in the audible query, the step of identifying atleast one data operation including identifying the at least one dataoperation based on both the identified intent and the qualifyingparameters.

In at least some cases at least one of the qualifying parameters caninclude a patient identity.

In at least some cases at least one of the qualifying parameters caninclude a patient’s disease state.

In at least some cases at least one of the qualifying parameters caninclude a genetic mutation.

In at least some cases at least one of the qualifying parameters caninclude a procedure type.

In at least some cases the method can further include identifyingqualifying parameters in the specific patient’s molecular report, thestep of identifying at least one data operation including identifyingthe at least one data operation based on both the identified intent andthe qualifying parameters.

In at least some cases the method can further include the step ofstoring a general knowledge database that includes non-patient specificdata about specific topics, wherein the step of identifying at least onedata operation associated with the at least one intent includesidentifying at least first and second data operations associated withthe at least one intent, the first data operation associated with thespecific patient’s molecular report and the second data operationassociated with the general knowledge database.

In at least some cases the second data operation associated with thegeneral knowledge database can be executed first to generate second dataoperation results, the second data operation results can be used todefine the first data operation and the first data operation associatedwith the specific patient’s molecular report can be executed second togenerate the first set of response data.

In at least some cases the first data operation associated with thespecific patient’s molecular report can be executed first to generatefirst data operation results, the first data operation results can beused to define the second data operation and the second data operationassociated with the general knowledge database can be executed second togenerate the first set of response data.

In at least some cases the step of identifying at least one intent caninclude determining that the audible query is associated with thespecific patient, accessing the specific patient’s molecular report,determining the specific patient’s cancer state from the molecularreport and then selecting an intent from a pool of cancer state relatedintents.

In at least some cases the method can further include the step ofstoring a general knowledge database that includes non-patient specificdata about specific topics, the method further including the steps of,upon determining that the audible query is not associated with anyspecific patient, selecting an intent that is associated with thegeneral knowledge database.

In at least some cases the collaboration device can include a portablewireless device that includes a wireless transceiver.

In at least some cases the collaboration device can be a handhelddevice.

In at least some cases the collaboration device can include at least onevisual indicator, the processor linked to the visual indicator andcontrollable to change at least some aspect of the appearance of thevisual indicator to indicate different states of the collaborationdevice.

In at least some cases the processor can be programmed to monitormicrophone input to identify a “wake up” phrase, the processormonitoring for the audible query after the wake up phrase is detected.

In at least some cases a series of audible queries can be received viathe microphone, and the at least one of the identified data operationscan include identifying a subset of data that is usable with subsequentaudio queries to identify intents associated with the subsequentqueries.

In at least some cases the method can further include the steps of,based on at least one audible query received via the microphone andrelated data in a system database, identifying at least one activitythat a collaboration device user may want to perform and initiating theat least one activity.

In at least some cases the step of initiating the at least one activitycan include generating a second audible response file and broadcastingthe second audible response file to the user seeking verification thatthe at least one activity should be performed and monitoring themicrophone for an affirmative response and, upon receiving anaffirmative response, initiating the at least one activity.

In at least some cases the at least one activity can includeperiodically capturing health information from electronic health recordsincluded in the system database.

In at least some cases the at least one activity can include checkingstatus of an existing clinical or lab order.

In at least some cases the at least one activity can include ordering anew clinical or lab order.

In at least some cases the collaboration device can be one of asmartphone, a tablet computer, a laptop computer, a desktop computer, ora device sold under the trademark AMAZON ECHO.

In at least some cases the step of initiating the at least one activitycan include automatically initiating the at least one activity withoutany initiating input from the user.

In at least some cases the method can further including storing andmaintaining a general cancer knowledge database, persistently updatingthe specific patient’s molecular report, automatically identifying atleast one intent and associated data operation related to the generalcancer knowledge database based on the specific patient’s molecularreport data, persistently executing the associated data operation on thegeneral cancer knowledge database to generate a new set of response datanot previously generated and, upon generating a new set of responsedata, using the new set of response data to generate another audibleresponse file and broadcasting the another audible response file via thespeaker.

In at least some cases the method can also be used with an electronichealth records system that maintains health records associated with aplurality of patients including the specific patient, the method furtherincluding identifying at least another data operation associated withthe at least one intent and executing the another data operation on thespecific patient’s health record to generate additional response data.

In at least some cases the step of using the first set of response datato generate an audible response file can include using the response dataand the additional response data to generate the audible response file.

In at least some embodiments, a method of audibly broadcasting responsesto a user based on user queries about a specific patient’s molecularreport, the method for use with a collaboration device that includes aprocessor and a microphone and a speaker linked to the processor isprovided by the disclosure. The method includes storing a separatemolecular report for each of a plurality of patients in a systemdatabase, storing a general cancer knowledge database that includesnon-patient specific data about cancer topics, receiving an audiblequery from the user via the microphone, identifying at least one intentassociated with the audible query, identifying at least a first dataoperation associated with the at least one intent and the specificpatient’s molecular report, identifying at least a second data operationassociated with the at least one intent and the general cancer knowledgedatabase, accessing the specific patient’s molecular report and thegeneral cancer knowledge database, executing the at least a first dataoperation on a first set of data included in the specific patient’smolecular report to generate a first set of response data, executing theat least a second data operation of the general cancer knowledgedatabase to generate a second set of response data, using at least oneof the first and second sets of response data to generate an audibleresponse file, and broadcasting the audible response file via thespeaker.

In at least some embodiments, a method of audibly broadcasting responsesto a user based on user queries about a specific patient’s molecularreport, the method for use with a collaboration device that includes aprocessor and a microphone and a speaker linked to the processor isprovided by the disclosure. The method includes storing molecularreports for a plurality of patients in a system database, receiving anaudible query from the user via the microphone, determining that theaudible query is associated with the specific patient, accessing thespecific patient’s molecular report, determining the specific patient’scancer state from the molecular report, identifying at least one intentfrom a pool of intents related to the specific patient’s cancer stateand the audible query, identifying at least one data operationassociated with the at least one intent, executing at least one of theidentified at least one data operations on a first set of data includedin the specific patient’s molecular report to generate a first set ofresponse data, using the first set of response data to generate anaudible response file, and broadcasting the audible response file viathe speaker.

In at least some embodiments, a method of audibly broadcasting responsesto a user based on user queries about a patient, the method for use witha collaboration device that includes a processor and a microphone and aspeaker linked to the processor is provided by the disclosure. Themethod includes storing health records for a plurality of patients in asystem database and storing a general cancer knowledge database,receiving an audible query from the user via the microphone, identifyinga specific patient associated with the audible query, accessing thehealth records for the specific patient, identifying cancer related datain the specific patient/s health records, identifying at least oneintent related to the identified cancer related data, identifying atleast one data operation related to the at least one intent, executingthe at least one data operation on the general cancer knowledge databaseto generate a first set of response data, using the first set ofresponse data to generate an audible response file, and broadcasting theaudible response file via the speaker.

In at least some embodiments, a method of audibly broadcasting responsesto a user based on user queries about a specific patient molecularreport is provided by the disclosure. The method includes receiving anaudible query from the user to a microphone coupled to a collaborationdevice, identifying at least one intent associated with the audiblequery, identifying at least one data operation associated with the atleast one intent, associating each of the at least one data operationswith a first set of data presented on the molecular report, executingeach of the at least one data operations on a second set of data togenerate response data, generating an audible response file associatedwith the response data, and providing the audible response file forbroadcasting via a speaker coupled to the collaboration device.

In at least some cases the audible query can include a question about anucleotide profile associated with the patient.

In at least some cases the nucleotide profile associated with thepatient can be a profile of the patient’s cancer.

In at least some cases the nucleotide profile associated with thepatient can be a profile of the patient’s germline.

In at least some cases the nucleotide profile can be a DNA profile.

In at least some cases the nucleotide profile can be an RNA expressionprofile.

In at least some cases the nucleotide profile can be a mutationbiomarker.

In at least some cases the mutation biomarker can be a BRCA biomarker.

In at least some cases the audible query can include a question about atherapy.

In at least some cases the audible query can include a question about agene.

In at least some cases the audible query can include a question aboutclinical data.

In at least some cases the audible query can include a question about anext-generation sequencing panel.

In at least some cases can include a question about a biomarker.

In at least some cases the audible query can include a question about animmune biomarker.

In at least some cases the audible query can include a question about anantibody-based test.

In at least some cases the audible query can include a question about aclinical trial.

In at least some cases the audible query can include a question about anorganoid assay.

In at least some cases the audible query can include a question about apathology image.

In at least some cases the audible query can include a question about adisease type.

In at least some cases the at least one intent can be an intent relatedto a biomarker.

In at least some cases the biomarker can be a BRCA biomarker.

In at least some cases the at least one intent can be an intent relatedto a clinical condition.

In at least some cases the at least one intent can be an intent relatedto a clinical trial.

In at least some cases the at least one intent can include a drug intentrelated to a drug.

In at least some cases the drug intent can be related to chemotherapy.

In at least some cases the drug intent can be an intent related to aPARP inhibitor.

In at least some cases the at least one intent can be related to a gene.

In at least some cases the at least one intent can be related toimmunology.

In at least some cases the at least one intent can be related to aknowledge database.

In at least some cases the at least one intent can be related to testingmethods.

In at least some cases the at least one intent can be related to a genepanel.

In at least some cases the at least one intent can be related to areport.

In at least some cases the at least one intent can be related to anorganoid process.

In at least some cases the at least one intent can be related toimaging.

In at least some cases the at least one intent can be related to apathogen.

In at least some cases the at least one intent can be related to avaccine.

In at least some cases the at least one data operation can include anoperation to identify at least one treatment option.

In at least some cases the at least one data operation can include anoperation to identify knowledge about a therapy.

In at least some cases the at least one data operation can include anoperation to identify knowledge related to at least one drug.

In at least some cases the at least one data operation can include anoperation to identify knowledge related to mutation testing.

In at least some cases the at least one data operation can include anoperation to identify knowledge related to mutation presence.

In at least some cases the at least one data operation can include anoperation to identify knowledge related to tumor characterization.

In at least some cases the at least one data operation can include anoperation to identify knowledge related to testing requirements.

In at least some cases the at least one data operation can include anoperation to query for definition information.

In at least some cases the at least one data operation can include anoperation to query for expert information.

In at least some cases the at least one data operation can include anoperation to identify information related to recommended therapy.

In at least some cases the at least one data operation can include anoperation to query for information relating to a patient.

In at least some cases the at least one data operation can include anoperation to query for information relating to patients with one or moreclinical characteristics similar to the patient.

In at least some cases the at least one data operation can include anoperation to query for information relating to patient cohorts.

In at least some cases the at least one data operation can include anoperation to query for information relating to clinical trials.

In at least some cases the at least one data operation can include anoperation to query about a characteristic relating to a genomicmutation.

In at least some cases the characteristic can be loss of heterozygosity.

In at least some cases the characteristic can reflect the source of themutation.

In at least some cases the source can be germline.

In at least some cases the source can be somatic.

In at least some cases the characteristic can include whether themutation is a tumor driver.

In at least some cases the first set of data can include a patient name.

In at least some cases the first set of data can include a patient age.

In at least some cases the first set of data can include anext-generation sequencing panel.

In at least some cases the first set of data can include a genomicvariant.

In at least some cases the first set of data can include a somaticgenomic variant.

In at least some cases the first set of data can include a germlinegenomic variant

In at least some cases the first set of data can include a clinicallyactionable genomic variant.

In at least some cases the first set of data can include a loss offunction variant.

In at least some cases the first set of data can include a gain offunction variant.

In at least some cases the first set of data can include an immunologymarker.

In at least some cases the first set of data can include a tumormutational burden.

In at least some cases the first set of data can include amicrosatellite instability status. In at least some cases the first setof data can include a diagnosis.

In at least some cases the first set of data can include a therapy.

In at least some cases the first set of data can include a therapyapproved by the U.S. Food and Drug Administration.

In at least some cases the first set of data can include a drug therapy.

In at least some cases the first set of data can include a radiationtherapy.

In at least some cases the first set of data can include a chemotherapy.

In at least some cases the first set of data can include a cancervaccine therapy.

In at least some cases the first set of data can include an oncolyticvirus therapy.

In at least some cases the first set of data can include animmunotherapy.

In at least some cases the first set of data can include a pembrolizumabtherapy.

In at least some cases the first set of data can include a CAR-Ttherapy.

In at least some cases the first set of data can include a protontherapy.

In at least some cases the first set of data can include an ultrasoundtherapy.

In at least some cases the first set of data can include a surgery.

In at least some cases the first set of data can include a hormonetherapy.

In at least some cases the first set of data can include an off-labeltherapy.

In at least some cases the first set of data can include an on-labeltherapy.

In at least some cases the first set of data can include a bone marrowtransplant event.

In at least some cases the first set of data can include a cryoablationevent.

In at least some cases the first set of data can include aradiofrequency ablation.

In at least some cases the first set of data can include a monoclonalantibody therapy.

In at least some cases the first set of data can include an angiogenesisinhibitor.

In at least some cases the first set of data can include a PARPinhibitor.

In at least some cases the first set of data can include a targetedtherapy.

In at least some cases the first set of data can include an indicationof use.

In at least some cases the first set of data can include a clinicaltrial.

In at least some cases the first set of data can include a distance to alocation conducting a clinical trial.

In at least some cases the first set of data can include a variant ofunknown significance.

In at least some cases the first set of data can include a mutationeffect.

In at least some cases the first set of data can include a variantallele fraction.

In at least some cases the first set of data can include a low coverageregion.

In at least some cases the first set of data can include a clinicalhistory.

In at least some cases the first set of data can include a biopsyresult.

In at least some cases the first set of data can include an imagingresult.

In at least some cases the first set of data can include an MRI result.

In at least some cases the first set of data can include a CT result.

In at least some cases the first set of data can include a therapyprescription.

In at least some cases the first set of data can include a therapyadministration.

In at least some cases the first set of data can include a cancersubtype diagnosis.

In at least some cases the first set of data can include a cancersubtype diagnosis by RNA class.

In at least some cases the first set of data can include a result of atherapy applied to an organoid grown from the patient’s cells.

In at least some cases the first set of data can include a tumor qualitymeasure.

In at least some cases the first set of data can include a tumor qualitymeasure selected from at least one of the set of PD-L1, MMR, tumorinfiltrating lymphocyte count, and tumor ploidy.

In at least some cases the first set of data can include a tumor qualitymeasure derived from an image analysis of a pathology slide of thepatient’s tumor.

In at least some cases the first set of data can include a signalingpathway associated with a tumor of the patient.

In at least some cases the signaling pathway can be a HER pathway.

In at least some cases the signaling pathway can be a MAPK pathway.

In at least some cases the signaling pathway can be a MDM2-TP53 pathway.

In at least some cases the signaling pathway can be a PI3K pathway.

In at least some cases the signaling pathway can be a mTOR pathway.

In at least some cases the at least one data operations can include anoperation to query for a treatment option, the first set of data caninclude a genomic variant, and the associating step can includeadjusting the operation to query for the treatment option based on thegenomic variant.

In at least some cases the at least one data operations can include anoperation to query for a clinical history data, the first set of datacan include a therapy, and the associating step can include adjustingthe operation to query for the clinical history data element based onthe therapy.

In at least some cases the clinical history data can be medicationprescriptions, the therapy can be pembrolizumab, and the associatingstep can include adjusting the operation to query for the prescriptionof pembrolizumab.

In at least some cases the second set of data can include clinicalhealth information.

In at least some cases the second set of data can include genomicvariant information.

In at least some cases the second set of data can include DNA sequencinginformation.

In at least some cases the second set of data can include RNAinformation.

In at least some cases the second set of data can include DNA sequencinginformation from short-read sequencing.

In at least some cases the second set of data can include DNA sequencinginformation from long-read sequencing.

In at least some cases the second set of data can include RNAtranscriptome information.

In at least some cases the second set of data can include RNAfull-transcriptome information.

In at least some cases the second set of data can be stored in a singledata repository.

In at least some cases the second set of data can be stored in aplurality of data repositories.

In at least some cases the second set of data can include clinicalhealth information and genomic variant information.

In at least some cases the second set of data can include immunologymarker information.

In at least some cases the second set of data can include microsatelliteinstability immunology marker information.

In at least some cases the second set of data can include tumormutational burden immunology marker information.

In at least some cases the second set of data can include clinicalhealth information including one or more of demographic information,diagnostic information, assessment results, laboratory results,prescribed or administered therapies, and outcomes information.

In at least some cases the second set of data can include demographicinformation including one or more of patient age, patient date of birth,gender, race, ethnicity, institution of care, comorbidities, and smokinghistory.

In at least some cases the second set of data can include diagnosisinformation including one or more of tissue of origin, date of initialdiagnosis, histology, histology grade, metastatic diagnosis, date ofmetastatic diagnosis, site or sites of metastasis, and staginginformation.

In at least some cases the second set of data can include staginginformation including one or more of TNM, ISS, DSS, FAB, RAI, and Binet.

In at least some cases the second set of data can include assessmentinformation including one or more of performance status comprising atleast one of ECOG status or Karnofsky status, performance status score,and date of performance status.

In at least some cases the second set of data can include laboratoryinformation including one or more of types of lab, lab results, labunits, date of lab service, date of molecular pathology test, assaytype, assay result, molecular pathology method, and molecular pathologyprovider.

In at least some cases the second set of data can include treatmentinformation including one or more of drug name, drug start date, drugend date, drug dosage, drug units, drug number of cycles, surgicalprocedure type, date of surgical procedure, radiation site, radiationmodality, radiation start date, radiation end date, radiation total dosedelivered, and radiation total fractions delivered.

In at least some cases the second set of data can include outcomesinformation including one or more of Response to Therapy, RECIST score,Date of Outcome, date of observation, date of progression, date ofrecurrence, adverse event to therapy, adverse event date ofpresentation, adverse event grade, date of death, date of lastfollow-up, and disease status at last follow up.

In at least some cases the second set of data can include informationthat has been de-identified in accordance with a de-identificationmethod permitted by HIPAA.

In at least some cases the second set of data can include informationthat has been de-identified in accordance with a safe harborde-identification method permitted by HIPAA.

In at least some cases the second set of data can include informationthat has been de-identified in accordance with a statisticalde-identification method permitted by HIPAA.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with a cancer condition.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with a cardiovascularcondition.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with a diabetes condition.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with an autoimmune condition.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with a lupus condition.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with a psoriasis condition.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with a depression condition.

In at least some cases the second set of data can include clinicalhealth information of patients diagnosed with a rare disease.

In at least some cases the method can be performed in conjunction with adigital and laboratory health care platform.

In at least some cases the digital and laboratory health care platformcan generate a molecular report as part of a targeted medical careprecision medicine treatment.

In at least some cases the method can operate on one or moremicro-services.

In at least some cases the method can be performed in conjunction withone or more microservices of an order management system.

In at least some cases the method can be performed in conjunction withone or more microservices of a medical document abstraction system.

In at least some cases the method can be performed in conjunction withone or more microservices of a mobile device application.

In at least some cases the method can be performed in conjunction withone or more microservices of a prediction engine.

In at least some cases the method can be performed in conjunction withone or more microservices of a cell-type profiling service.

In at least some cases the method can be performed in conjunction with avariant calling engine to provide information to a query involvingvariants.

In at least some cases the method can be performed in conjunction withan insight engine.

In at least some cases the method can be performed in conjunction with atherapy matching engine.

In at least some cases the method can be performed in conjunction with aclinical trial matching engine.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

FIG. 1 is a schematic diagram illustrating a collaboration system thatis consistent with at least some aspects of the present disclosure thatincludes a portable wireless collaboration device;

FIG. 2 is a schematic illustrating components of the exemplarycollaboration device shown in FIG. 1 ;

FIG. 3 is a schematic diagram of a second exemplary collaborationdevice;

FIG. 4 is a schematic diagram illustrating components of the secondexemplary collaboration device shown in FIG. 3 ;

FIG. 5 is a flow chart illustrating a collaboration process that isconsistent with at least some aspects of the present disclosure;

FIG. 6 is a schematic illustrating a collaboration device user having aconversation with the system of claim 1;

FIG. 7 is a schematic illustrating a workstation usable to access storedcollaboration session data;

FIG. 8 is similar to FIG. 7 , albeit illustrating another screen shot;

FIG. 9 is a schematic illustrating a portable audible collaborationdevice being used in conjunction with a workstation including a display;

FIG. 10 is a schematic illustrating another screen shot that is similarto the FIG. 8 view; and

FIG. 11 is a schematic illustrating a second collaboration system thatis consistent with at least some aspects of the present disclosure,albeit where a portable collaboration device runs AI applications togenerate seed data for data operations and also converts data responsesto audio response files to be broadcast via the collaboration device;

FIG. 12 is a schematic illustrating a third collaboration system that isconsistent with at least some aspects of the present disclosure;

FIG. 13 is a schematic illustrating a number of collaboration devicesthat can communicate using mesh networking with each other and/or withat least one of a first transceiver and a second transceiver;

FIG. 14 shows two additional collaboration device configurationsincluding a cube shaped configuration and a tablet type configuration;

FIG. 15 is a schematic illustrating a workstation that includes varioustypes of input/output collaboration devices;

FIG. 16 illustrates a headset that may operate as yet another type ofinput/output audio interface that is consistent with at least someaspects of the present disclosure;

FIGS. 17A-17C are schematics showing a first through third pages of apancreatic clinical report that may be printed in hardcopy or accessedelectronically via a workstation, pad or smart phone device, etc.;

FIG. 18 is a flowchart similar to the chart shown in FIG. 5 , albeitwhere a state-specific clinical record and related intents are used todrive a query process;

FIG. 19 is an audio response process that is consistent with at leastsome aspects of the present disclosure;

FIG. 20 is a system database that is consistent with at least someaspects of the present disclosure;

FIG. 21 is a screen shot for use by a system administrator forspecifying system intents, intent parameters and answer formats forprovider panel types that is consistent with at least some aspects ofthe present disclosure;

FIG. 22 is similar to FIG. 21 , albeit including a screen shot forspecifying gene specific system information;

FIG. 23 is similar to FIG. 22 , albeit including a screen shot forspecifying provider methods;

FIG. 24 is a schematic diagram of an exemplary fourth exemplary systemincluding a mobile device;

FIG. 25 is a screen shot of a mobile application;

FIG. 26 is a second screenshot of the mobile application in FIG. 25 ;

FIG. 27 is a third screenshot of the mobile application in FIG. 25 ;

FIG. 28 is a schematic diagram of a fifth exemplary collaborationsystem;

FIG. 29 is a flowchart of a process for generating supplemental contentfor a physician based on a molecular report associated with a specificpatient;

FIG. 30 is a flowchart of a process for generating non-patient-specificsupplemental content for a physician;

FIG. 31 is a flowchart of a process that may be used for onboarding anoncologist;

FIG. 32 is a screen shot for use by a system administrator for visuallyspecifying system intents, intent parameters and answer formats forprovider panel types that is consistent with at least some aspects ofthe present disclosure; and

FIG. 33 is a schematic diagram of an intent extraction architecture;

FIG. 34 is a schematic diagram of a question and answer workflow;

FIG. 35 is a schematic diagram of an exemplary conversation workflow;and

FIG. 36 is a flowchart of a process that provides an audible response toan oncologist using at least one microservice and/or engine that isconsistent with at least some aspects of the present disclosure.

DETAILED DESCRIPTION OF THE DISCLOSURE

The various aspects of the subject disclosure are now described withreference to the drawings, wherein like reference numerals correspond tosimilar elements throughout the several views. It should be understood,however, that the drawings and detailed description hereafter relatingthereto are not intended to limit the claimed subject matter to theparticular form disclosed. Rather, the intention is to cover allmodifications, equivalents, and alternatives falling within the spiritand scope of the claimed subject matter.

In the following detailed description, reference is made to theaccompanying drawings which form a part hereof, and in which is shown byway of illustration, specific embodiments in which the disclosure may bepracticed. These embodiments are described in sufficient detail toenable those of ordinary skill in the art to practice the disclosure. Itshould be understood, however, that the detailed description and thespecific examples, while indicating examples of embodiments of thedisclosure, are given by way of illustration only and not by way oflimitation. From this disclosure, various substitutions, modifications,additions rearrangements, or combinations thereof within the scope ofthe disclosure may be made and will become apparent to those of ordinaryskill in the art.

In accordance with common practice, the various features illustrated inthe drawings may not be drawn to scale. The illustrations presentedherein are not meant to be actual views of any particular method,device, or system, but are merely idealized representations that areemployed to describe various embodiments of the disclosure. Accordingly,the dimensions of the various features may be arbitrarily expanded orreduced for clarity. In addition, some of the drawings may be simplifiedfor clarity. Thus, the drawings may not depict all of the components ofa given apparatus (e.g., device) or method. In addition, like referencenumerals may be used to denote like features throughout thespecification and figures.

Information and signals described herein may be represented using any ofa variety of different technologies and techniques. For example, data,instructions, commands, information, signals, bits, symbols, and chipsthat may be referenced throughout the above description may berepresented by voltages, currents, electromagnetic waves, magneticfields or particles, optical fields or particles, or any combinationthereof. Some drawings may illustrate signals as a single signal forclarity of presentation and description. It will be understood by aperson of ordinary skill in the art that the signal may represent a busof signals, wherein the bus may have a variety of bit widths and thedisclosure may be implemented on any number of data signals including asingle data signal.

The various illustrative logical blocks, modules, circuits, andalgorithm acts described in connection with embodiments disclosed hereinmay be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and acts are described generally in terms of theirfunctionality. Whether such functionality is implemented as hardware orsoftware depends upon the particular application and design constraintsimposed on the overall system. Skilled artisans may implement thedescribed functionality in varying ways for each particular application,but such implementation decisions should not be interpreted as causing adeparture from the scope of the embodiments of the disclosure describedherein.

In addition, it is noted that the embodiments may be described in termsof a process that is depicted as a flowchart, a flow diagram, astructure diagram, or a block diagram. Although a flowchart may describeoperational acts as a sequential process, many of these acts can beperformed in another sequence, in parallel, or substantiallyconcurrently. In addition, the order of the acts may be re-arranged. Aprocess may correspond to a method, a function, a procedure, asubroutine, a subprogram, etc. Furthermore, the methods disclosed hereinmay be implemented in hardware, software, or both. If implemented insoftware, the functions may be stored or transmitted as one or moreinstructions or code on a computer-readable medium. Computer-readablemedia includes both computer storage media and communication mediaincluding any medium that facilitates transfer of a computer programfrom one place to another.

It should be understood that any reference to an element herein using adesignation such as “first,” “second,” and so forth does not limit thequantity or order of those elements, unless such limitation isexplicitly stated. Rather, these designations may be used herein as aconvenient method of distinguishing between two or more elements orinstances of an element. Thus, a reference to first and second elementsdoes not mean that only two elements may be employed there or that thefirst element must precede the second element in some manner. Also,unless stated otherwise a set of elements may comprise one or moreelements.

As used herein, the terms “component,” “system” and the like areintended to refer to a computer-related entity, either hardware, acombination of hardware and software, software, or software inexecution. For example, a component may be, but is not limited to being,a process running on a processor, a processor, an object, an executable,a thread of execution, a program, and/or a computer. By way ofillustration, both an application running on a computer and the computercan be a component. One or more components may reside within a processand/or thread of execution and a component may be localized on onecomputer and/or distributed between two or more computers or processors.

The word “exemplary” is used herein to mean serving as an example,instance, or illustration. Any aspect or design described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other aspects or designs.

Furthermore, the disclosed subject matter may be implemented as asystem, method, apparatus, or article of manufacture using standardprogramming and/or engineering techniques to produce software, firmware,hardware, or any combination thereof to control a computer or processorbased device to implement aspects detailed herein. The term “article ofmanufacture” (or alternatively, “computer program product”) as usedherein is intended to encompass a computer program accessible from anycomputer-readable device, carrier, or media. For example, computerreadable media can include but are not limited to magnetic storagedevices (e.g., hard disk, floppy disk, magnetic strips ...), opticaldisks (e.g., compact disk (CD), digital versatile disk (DVD) .. .),smart cards, and flash memory devices (e.g., card, stick). Additionallyit should be appreciated that a carrier wave can be employed to carrycomputer-readable electronic data such as those used in transmitting andreceiving electronic mail or in accessing a network such as the Internetor a local area network (LAN). Of course, those skilled in the art willrecognize many modifications may be made to this configuration withoutdeparting from the scope or spirit of the claimed subject matter.

The term “genetic analyzer” is used herein to mean a device, system,and/or methods for determining the characteristics (including sequences)of nucleic acid molecules (including DNA, RNA, etc.) present inbiological specimens (including tumors, biopsies, tumor organoids, bloodsamples, saliva samples, or other tissues or fluids).

The term “genetic profile” is used herein to mean a combination of oneor more variants, RNA transcriptomes, or other informative geneticcharacteristics determined for a patient from next-generationsequencing. The next-generation sequencing may also be commonly referredto as “massively parallel sequencing.”

The term “genetic sequence” is used herein to mean a recordation of aseries of nucleotides present in a patient’s RNA or DNA as determinedfrom sequencing the patient’s tissue or fluids.

The term “variant” is used herein to mean a difference in a geneticsequence or genetic profile when compared to a reference geneticsequence or expected genetic profile.

The term “expression level” is used herein to mean the number of copiesof an RNA or protein molecule generated by a gene or other geneticlocus, which may be defined by a chromosomal location or other geneticmapping indicator.

The term “gene product” is used herein to mean a molecule (including aprotein or RNA molecule) generated by the manipulation (includingtranscription) of the gene or other genetic locus, which may be definedby a chromosomal location or other genetic mapping indicator.

Referring now to the drawings wherein like reference numerals correspondto similar elements throughout the several views and, more specifically,referring to FIG. 1 , the present disclosure will be described in thecontext of an exemplary collaboration system 10 that is consistent withat least some aspects of the present disclosure. System 10 includes acollaboration server 12, an artificial intelligence (AI) server 14, auser interface collaboration device 20 and a service provider database18. Referring again to FIG. 1 , in the illustrated embodiment AI server14 is shown as separate from collaboration server 12. Nevertheless, itshould be appreciated that in at least some embodiments the functionsthe two servers may be performed via a single server. Similarly, whileexemplary system 10 is described herein as one where specific processsteps or functions are performed by server 12 and others are performedby server 14, in other cases division of the functions and steps betweenthe two servers 12 and 14 may be different. Furthermore, in at leastsome embodiments some of the processes performed by the servers 12 and14 may be performed by a processor located within collaboration device20. For instance, in at least some cases, some or most of the processesrelated to speech recognition, intent matching, parameter extraction andaudio response generation performed by AI server 14 may be performed bycollaboration device 20. Having at least some of the processes performedby servers 12 and 14 performed on the collaboration device 20 can reducelatency of outputting a generated response by up to two seconds, andwill be explained in further detail below.

Collaboration server 12 is linked to a wireless transceiver (e.g.,transmitter and receiver) 16 enabling wireless two-way communicationbetween collaboration device 20 and collaboration server 12. Transceiver16 may be any type of wireless transceiver including, for instance, acellular phone transceiver, a WIFI transceiver, a BLUETOOTH transceiver,a combination of different types of transceivers (e.g., includingBLUETOOTH and cellular), etc. Server 12 runs software applications ormodules to perform various processes and functions described throughoutthis specification. In particular, server 12 runs a collaborationapplication 60 which includes, among other things, a visual responsemodule 62 and a data operation module 64. Server 12 receives user voicequeries (hereinafter “voice messages”) 59 captured by device 20,cooperates with AI server 14 to identify the meaning (e.g., intent andimportant parameters) of the voice messages, runs data operations ondata in database 18 that is consistent with the voice messages togenerate data responses, cooperates with AI server 14 to generate audioresponse files based on the data responses and, in at least some casesvisual response files, and transmits the response files 73, 77 back tocollaboration device 20. Device 20 in turn broadcasts 66 an audioresponse to the user and in cases where there is a visual responsesuitable for presentation via device 20, generates the visual responsein some fashion (e.g., presents content on a device 20 display 48,illuminates a signaling light 50, etc.). The display 48 and/or signalinglight 50 may be considered a visual indicator(s).

Referring also to FIG. 2 , collaboration device 20 includes an externalhousing 22, a device processor 30, a battery 32 or other power source, adevice memory 34, a wireless transceiver 36, one or more microphones 38and one or more speakers 44 or audio output devices, as well as somecomponent or process that can be used to activate device 20 to initiatea user collaboration activity. External housing 22 includes an externalsurface that forms a sphere in the illustrated example where a diameterof the sphere is selected so that the device 20 can easily be held byhand by an oncologist. For instance, the diameter of device 20 in mostcases will be between three fourths of an inch and five inches and inparticularly advantageous embodiments the diameter will be between oneand one quarter inch and two inches.

In other cases the external housing includes an external surface thatforms a cube or other three-dimensional rectangular prism. In suchcases, in particularly advantageous embodiments, the largest dimensionof the three-dimensional shape (height, width, depth) will be betweenone and one quarter inch and two inches.

The system 10 may be implemented in other manners. For instance, thecollaboration device 20 may be a smartphone, tablet, laptop, desktop, orother computing device, such as an APPLE IPHONE, a smartphone runningthe ANDROID operating system, or an AMAZON ECHO. Some of the processesperformed by the servers 12 and 14 may be performed through the use ofan app or another program executed on a processor located withincollaboration device 20.

The outside surface may be formed by several different components out ofseveral different materials including opaque materials for some portionsof the surface and transparent or translucent materials for otherportions where light needs to pass from indicator lights mounted withinthe housing. The outside housing surface may form speaker and microphoneapertures, a charging port opening (not illustrated), and otherapertures or openings for different purposes. The housing forms aninternal cavity and most of the other device components are mountedtherein. While device 20 may include a single speaker and a singlemicrophone, in an optimized assembly device 20 will include severalspeakers and microphones arrayed about the housing assembly so thatoncologist voice signals can be picked up from all directions.

There are many different hardware configurations that may be used toprovide the collaboration device processor 30. One particularly usefulprocessor for purposes of the present device 30 is the QUALCOMM QCS405SoC (System-on-Chip) which supports many different types of connectivityincluding BLUETOOTH, ZIGBEE, 802.11ac, 802.11ax-ready, USBC2.0/3.0, andothers. This solution includes an on device AI engine that enables ondevice AI algorithm execution so that, in at least some cases, the AIfunctionality described herein in relation to server 14 may be performedby processor 30. This SoC supports up to four microphones and supportshigh performance key word detection. Processor 30 is linked to each ofbattery 32, memory 34, transceiver 36, microphone 38, and speaker 44. Insome embodiments, the battery 32 can be charged using a charging dock(not shown).

Once device 20 is activated and while it remains active, microphone 38captures user voice messages 57 which are provided to processor 30.Processor 30 transmits 59 the voice messages via transceiver 36 tocollaboration server 12 (see again FIG. 1 ). Audio response files arereceived 81 back by device 20 transceiver 36 and processor 30 broadcaststhose response files via speaker 44. While not shown it is contemplatedthat device 20 may also include some type of haptic signaling component(e.g., a vibrator or the like) to indicate one or more device states.

The device 20, and more specifically, the memory 34, can includeacoustics processes, light control processes, security processes,connectivity processes, and other suitable purposes. These processes canbe stored as firmware on a portion of the memory 34 that isnon-volatile, and in some embodiments, read-only. The firmware caninclude acoustics processes that can recognize a library of wake wordsor phrases the oncologist enunciates. For example, the library caninclude the phrases “TEMPUS One” or “Hey ONE.” Having key phrases thatthe oncologist will repeatedly use stored directly on the memory 34 canreduce latency in processing commands the oncologist enunciates. Theacoustics processes can also include silence detection processes,fallback audio response playback processes that audibly notify theoncologist of errors or time-outs that occur during data transmissions(e.g., of TCP packets or HTTP messages), speaker protection algorithms,digital signal processing (DSP) algorithms, and/or other suitableprocesses related to acoustics.

The firmware can include conversational flow processes for determiningwhether a follow-up question would require the use of a wake word phrasesuch as “TEMPUS ONE.” For example, an initial question of “What wereDwayne Holder’s results?” could be followed by “And how old was he?”without using the “TEMPUS ONE” wake word phrase or specifying the nameof the patient again. The question “And how old was he?” may not berelevant unless the person in question has already been identified, inwhich case the follow-up question can be asked. The firmware can includebattery status algorithms for determining charge levels and/or chargestatus of the battery 32. The firmware can include connectivity andsecurity processes storing and/or maintaining crypto keys for a secureelement, storing device identifiers, storing valid networks (e.g., WiFinetworks), and other suitable processes. In some embodiments, thefirmware can be updated over the air.

The firmware can also include lighting control processes for controllingthe indicator lights 50. The lighting control processes can change thecolor and/or brightness of the indicator lights 50, as well as pulse theindicator lights 50 on and off.

The firmware processes can be used to control the indicator lights 50and/or speakers 44 based on a state of the collaboration device 20. Somestates are initiated by the oncologist. The oncologist can actuate oneor more of the input buttons 52, enunciate commands, and/or move thecollaboration device 20 (e.g., by setting the collaboration device 20 onthe charging dock). Exemplary lighting and speaker controls based on thestate of the collaboration device 20 are included in Table 1 below. Someoncologist interactions include “app” functionality, which will beexplained in detail below.

TABLE 1 CATEGORY STATE INDICATO R LIGHT EFFECT INDICATO R LIGHT COLORSOUND USER INPUT POWER Turn On Chasing Clockwise 1x White Ascending Bootup Push input button Turn Off Chasing Counterclock 1x White DescendingPower Down Push input button CONNECTIVI TY WiFi Pairing Pulse BlueSelect WiFi Network (in app) WiFi Connected 3 Blips Blue Success!Notification (in app) WiFi Not Connected Pulse 3x Red Try Again/ SeeTroubleshooti ng QUERY or COMMAND Wake & Listen Chasing Clockwise 1xBlue “TEMPUS One” Query or Command Solid Blue Ask question or GivingCommand QUERY RESPONSE Thinking Fast Pulse Blue Responding Solid Blue“Answer...” Query Unsuccessful Solid Blue “I’m sorry, please ask again.”COMMANDS Precede with: “TEMPUS One, ______” Volume Up/Down Solid White...”Volume Up″ “Volume Down” “Volume 1-10” Stop ... “Stop” Power Off...”Turn off” “Power off” Battery Status ...”Battery level” BATTERYPower On while battery is low Blink 2x Red “Battery Low” At end ofResponse Blink 2x Red “Battery Low” while battery is low Charging on thedock Breathing pulse Red (critical low) / White Short chime when setdown Set on the charging dock Fully charged while sitting on dock SolidWhite ERROR Any error/alert that needs app intervention Pulse RedConsult App

In at least some cases device 20 can be activated by a specificallyuttered voice command. To this end, processor 30 may be on all the timeand monitoring for a special triggering activation command like “HeyQuery”. Once the activation command is received, processor 30 may beactivated to participate in a user collaboration session. Here,processor 30 may acknowledge the activation command by transmitting aresponse like “Hello, what can I help you with?” or a tone or otheraudio indication, and may then enter a “listening” state to capture asubsequent user voice message. When a subsequent voice message iscaptured, the collaboration session may proceed as described above.

In addition to or instead of being activated by an uttered activationcommand, device 20 may be activated by selection of a device activationbutton or touch sensor, when the device 20 is picked up or otherwisemoved, etc. To this end, see the optional input buttons 52 and motionand orientation sensors 40 and 42 in FIG. 2 . The motion sensors mayinclude an accelerometer, a gyroscope, both an accelerometer and agyroscope or some other type of motion sensor device.

In addition to being able to present audio responses to a user’squeries, in at least some cases device 20 is equipped to present sometype of visual response. For instance in a simple case, device 20 mayinclude more or more indicator lights 50 where LED or other lightsources can be activated or controlled to change colors to intricatedifferent device 20 states. For instance, in at least some casesindicator lights 50 may be off or dimmed green when device 20 isinactive and waiting to be activated. Here, once device 20 is activatedand while waiting or listening for a voice message, lights 50 may beactivated bright green to indicate “go”. As a user is speaking and thevoice message is being captured by device 20, lights 50 may be activatedblue green to indicate an audio message capture state. Once a queryvoice signal has ended, lights 50 may be illuminated yellow indicating a“thinking” or query processing state. As an audio response is beingbroadcast to the user, lights 50 may be illuminated orange to indicatean output state and once the audio response is complete, lights 50 mayagain be illuminated bright green to indicate that device is againwaiting or listening for a next voice message to be uttered by the user.

In at least some cases any time device 20 is activated and waiting for anew or next voice message, device 20 may be programmed to wait in theactive state for only a threshold duration (e.g., 30 seconds) and thenassume an inactive state waiting to be re-activated via anotheractivation utterance or other user input. In other cases, once device 20is activated, it may remain activated for a longer duration (e.g., 10minutes) and only enter the deactivated listening state prior to the endof the longer duration if a user utters a deactivation phrase (e.g.,“End session”, “End query” or “Hey query” followed by “End session”) orotherwise affirmatively deactivates the device 20 (e.g., selects adeactivation input button 52).

Referring still to FIG. 2 , in some cases, device 20 may include one ormore flat or curved or otherwise contoured display screens 48 forpresenting visual responses to user queries where the visual responsesare suitable for consumption via a relatively small display screen.Here, for instance, short answers to user queries may be presented astext via display 48. As another instance, summary phrases related todata responses that include data that cannot easily be presented via asmall display screen may be generated and presented via display 48.Other text phrases or graphics are contemplated for other purposes. Forinstance, in cases where a visual response is presented via some otherdisplay device (e.g., a display device that is paired or otherwiseassociated with collaboration device 20), a text message may bepresented via display 48 indicating that additional information or avisual response is being presented via the associated display. Asanother instance, display 48 may be controlled to glow specific colorsto indicate states as described above with respect to light devices 50and may only present answers to queries in a textual format. Referringagain to FIG. 1 , AI server 14 runs software application programs andmodules that perform various functions consistent with at least someaspects of the present disclosure. In at least some cases, AI server 14includes an automatic speech recognition (ASR) module 70, an intentmatching module 72, a parameter extraction module 74 and an audioresponse module 76.

ASR module 70 receives 61 user voice messages from collaborationapplication 60 and automatically converts the voice signals to textcorresponding to the user’s uttered voice messages essentially in realtime. Thus, if an oncologist’s voice signal message is “How many malepatients 45 years or older have had pancreatic cancer?” or “What type oftreatment should I prescribe this patient?”, ASR module 70 generatesmatching text using speech recognition software. Speech recognitionapplications are well known in the art and include DRAGON software byNUANCE, GOOGLE VOICE by GOOGLE, and WATSON by IBM, as well as others. Insome cases recognition applications support industry specificterm/phrase lexicons where specific terms and phrases used within theindustries are defined and recognizable. In some cases user specificlexicons are also supported for terms or phrases routinely used byspecific oncologists. In each of these cases new terms and phrases canbe added to the industry and user lexicons. The text files are providedto intent matching module 72.

Intent matching module 72 includes a natural language processor (NLP)that is programmed to determine an intent of the user’s voice signalmessage. Here, for instance, the intent may be to identify a data subsetin database 18. As another instance, the intent associated with thephrase “How many male patients 45 years or older have had pancreaticcancer?” may be to identify a number of patients. As another example,the intent associated with the phrase “What type of treatment should Iprescribe patient John Doe?” may be to identify the treatment that thesystem determines will maximally extend the quality of life for thepatient John Doe. Literally thousands of other intents may be discernedby matching module 72. Intents are described in greater detailhereafter.

Referring again to FIG. 1 , parameter extraction module 74 extractsimportant parameters from the user’s uttered voice message. Forinstance, extracted parameters from the phrase “How many male patients45 years or older have had pancreatic cancer?” may include “pancreatic”,“male” and “45 years”. For each user voice message, AI server 14provides 63 (i) the associated text file, (ii) the matching intent and(iii) the extracted parameters back to collaboration server 12 and morespecifically to the data operation module 64.

Data operation module 64 accesses database 18 and creates 65 acollaboration record on the database to memorialize the collaborationsession. The text file received from server 14 is stored in database 18along with a date and time, oncologist identifying information, etc.Data operation module 64 converts the intent and extracted parametersinto a data operation and then performs 65 the operation on data indatabase 18. For instance, in the case of the voice message “How manymale patients 45 years or older have had pancreatic cancer?”, operationmodule 64 structures a database query to search for a number (e.g., theintent) of male patients 45 or older that had pancreatic cancer (e.g.,the extracted parameters). The data operation results in a data responseincluding the number of male patients 45 or older that had pancreaticcancer.

As another example, in the case of the voice message “What type ofmedication should I prescribe for John Doe,” operation module 64structures a database query to search for a medication (e.g., theintent) of a cohort of patients who are clinically similar to thepatient John Doe, where such medication resulted in an optimal outcomefor the cohort. The determination of whether a cohort of patients isclinically similar may be achieved by querying the database 18 forpatients with certain factors, such as age, cancer stage, priortreatments, variants, RNA expression, etc. that are the same and/orsimilar to those of John Doe. As a simple example, if John Doe has aPTEN genomic mutation, the database 18 may select for inclusion into thecohort all patients who also have a PTEN genomic mutation. As anotherexample, if John Doe has metastatic prostate cancer but no longerresponds to androgen suppression first line therapy, the database 18 mayselect for inclusion into the cohort all metastatic prostate cancerpatients who no longer responded to androgen suppression first linetherapy.

As another example, in the case of the voice message “What is theexpected progression free survival for Jane Smith if I prescribeKEYTRUDA,” operation module 64 structures a database query to search forpatients clinically similar to Jane Smith; selects from those patients acohort who were prescribed KEYTRUDA; analyzes the progression freesurvival of the selected cohort of patients; and returns the averageprogression free survival from the selected cohort.

As indicated above, the physician’s voice message may relate to aquestion about a particular individual. The operation module 64 mayfurther be arranged to access a patient data repository in order toidentify clinical, genomic, or other health information of the patient.The patient data repository may take many forms, and may include anelectronic health record, a health information exchange platform, apatient data warehouse, a research database, or the like. The patientdata repository may include data stored in structured format, such as arelational database, JSON files, or other data storage arrangementsknown in the art. The operation module 64 may communicate with thepatient data repository in various ways, such as through a dataintegration, may use various technologies, and may rely on variousframeworks, such as Fast Healthcare Interoperability Resources (FHIR).The patient data repository may be owned, operated, and/or controlled bythe physician, the physician’s employer, a hospital, a physicianpractice, a clinical laboratory, a contract research organization, oranother entity associated with the provision of health care. The patientdata repository may include all of the patient’s health information, ora subset of the patient’s health information. For instance, the patientdata repository may include structured data with patient demographicinformation (such as age, gender, etc.) a clinical description of thepatient’s cancer (for instance, a staging such as “stage 4” and asubtype such as “pancreatic cancer”, etc.), a genomic description of thepatient and/or the patient’s cancer (for instance, nucleotide listingsof certain introns or exons; somatic variants such as “BRAF mutation”;variant allele frequency; immunology markers such as microsatelliteinstability and tumor mutational burden; RNA overexpression orunderexpression; a listing of pathways affected by a found variant;etc.), an imaging description of the patient’s cancer (for instance,features derived from radiology or pathology images), anorganoid-derived description of the patient’s cancer (for instance, alisting of treatments that were effective in reducing or destroyingorganoid cells derived from the patient’s tumor), and a list of priorand current medications, therapies, surgeries, procedures, or othertreatments.

The operation module 64 may use various methods to identify how theparticular patient being queried about is clinically similar to otherindividuals whose data is stored in the database 18. Examples ofdetermining clinical similarity are described in U.S. Pat. PublicationNo. 2020/0135303, published Apr. 30, 2020, the contents of which areincorporated herein by reference in their entirety, for all purposes.Other examples of determining clinical similarity are described in U.S.Pat. Publication No. 2020/0211716, published Jul. 2, 2020, the contentsof which are incorporated herein by reference in their entirety, for allpurposes.

The determination of what medication resulted in an optimal outcome foran identified cohort of individuals may be determined by comparing theoutcome information stored in database 18 for those individuals with themedications that were prescribed or administered to them; dividing thecohort into sub-cohorts; analyzing, for each sub-cohort, measures ofoutcome such as progression-free survival, overall survival, quality ofsurvival, or so forth; and returning one or more measures that indicatethe optimal outcome(s).

In another example, the data operation module 64 may select a firsttreatment from a list of treatments; examine the information from allpatients in the database 18 who were provided that first treatment;divide the patient group into a first cohort of patients with a positiveoutcome and second cohort of patients without a positive outcome;compare the health characteristics (such as clinical, genomic, and/orimaging) of the queried patient to the health characteristics of thefirst cohort; compare the health characteristics of the queried patientto the health characteristics of the second cohort; and determinewhether the queried patient’s characteristics are closer to those of thefirst cohort or the second cohort. If the queried patient’scharacteristics are more clinically similar to the first cohort, thenthe data operation module 64 may prepare a data response indicating thefirst treatment. If the queried patient’s characteristics are moreclinically similar to the second cohort, then the data operation module64 may not prepare a date response indicating the first treatment. Thedata operation module 64 may then select a second, third, fourth, etc.treatment from the list of treatments and repeat the process describedabove for each selected treatment, and may continue until all treatmentsin the list of treatments have been explored. A variety of algorithmicapproaches using mathematical or statistical methods known in the artmay be used on the relevant health characteristics to determine whetherthe queried patient characteristics are clinically similar to the firstcohort or second cohort, including mean, median, principal componentanalysis, and the like.

In another example, the data operation module 64 may select all or asubset of records from patients in the database 18. From those records,the module 64 may then select records from a first cohort of patientswith a genomic biomarker similar to the queried patient. The module 64may then filter the first cohort for those patients who were prescribeda first treatment from a list of treatments. The module 64 may thenexamine the outcomes of the patients in the first cohort and subdividethe first cohort into two or more sub-cohorts based on outcome, withpatients with similar outcomes divided into the same sub-cohorts. Eachsub-cohort may be further divided like the first cohort into additionalsub-cohorts, and so on and so on until there is no material outcomesdifference within each sub-cohort. At this point in the method, theremay be dozens or more of sub-cohorts. The data operations module 64 maythen compare the queried patient’s health characteristics with those ineach sub-cohort, to identify the sub-cohort that is most clinicallysimilar to the patient’s health characteristics. The data operationmodule 64 may then select a second, third, fourth, etc. treatment fromthe list of treatments and repeat the process described above for eachselected treatment, and may continue until all treatments in the list oftreatments have been explored.

Data operation module 64 returns 67 the data response to AI server 14and, more specifically to audio response module 76, which uses that datato generate an audio response file. For instance, where 576 malepatients 45 years or older had pancreatic cancer in the datasetsearched, response module 76 may generate the phrase “576 male patents45 years or older have had pancreatic cancer.” The audio response fileis transmitted 71 back to collaboration application 60. Thecollaboration application stores the response file as well as a textualrepresentation thereof in the collaboration record on database 18 forsubsequent access. Collaboration application 60 also transmits 73 theaudio response file via transceiver 16 to collaboration device 20 whichthen broadcasts that audio file to the user.

The AI modules 14 may be provided via many different softwareapplication programs. One particularly useful suite of software modulesthat may provide the AI functionality is the QUALCOMM SMART AUDIO 400Platform Development Kit that can be used with the QUALCOMM SoCprocessor described above. Another useful suite is the DIALOGFLOWprogram developed and maintained by GOOGLE. DIALOGFLOW is an end-to-end,build-once deploy-everywhere development suite for creatingconversational interfaces for websites and mobile applications. A systemadministrator can use DIALOGFLOW interfaces to define a set of intents,training phrases, parameters, and responses to intents. An intent is ageneral intention (e.g., what a user wants) -by a user to access ormanipulate database data in some specific way. For instance, one intentmay be to generate a database data subset (e.g., patients that meetqualifying query parameters). As another instance, another intent may beto return a number (e.g., number of patients that meet qualifyingparameters). Other intents may be a welcome intent (e.g., when a userfirst activates device 20), an adverse consequences intent (e.g., toreturn a list of or at least an indication of adverse consequences to atreatment regimen), a medications intent (e.g., to return a list orindication prior medications), a schedule event intent (e.g., toschedule an appointment, a test, a procedure, etc.), etc. It isanticipated that a typical system will include hundreds and in somecases thousands of intents.

For each intent, the administrator provides a relatively small set ofseed or training phrases used to train the intent matching module torecognize an intent associated with a received voice message. Thetraining phrases include phrases that a user might say when theirobjective or purpose associated with an utterance is consistent with theassociated intent. For instance, for an intent to return a number ofpatients that meet qualifying parameters (e.g., age, ailment, condition,oncogene, mutation, residence, staging, treatment, adverse effects ofmedical YYY, outcomes, etc.), some exemplary training phrases may be“How many patients have pancreatic cancer?”, “How many stage 3 breastcancer patients from Chicago are HER2 positive?”, “What number ofpatients have shown adverse effects while taking medication XXX?”, “Howmany ovarian cancer patients in the last 48 months have had a p85 PIK3CAmutation?”, “What percentage of basal cell carcinoma patients in thelast 18 months have had cryosurgery?”, and “The number of people thatsmoke that also have lung cancer?” DIALOGFLOW also supports follow upintents that may be serially associated with other intents and morespecifically with a second or subsequent intent to be discerned in aseries of questions after a first intent is identified. For instance,the first phrase “How many ovarian cancer patients in the last 48 monthshave had a p85 PIK3CA mutation?” could be followed by a second phrase“How many of those patients were seen in the last 12 months?” As anotherexample, for an intent to return a suggested therapy for a specificpatient, some exemplary training phrases may be “What type of medicationshould I prescribe for John Doe?”, “What type of immunotherapy shouldthis patient receive”, “What is the expected progression free survivalfor Jane Smith if I prescribe KEYTRUDA?”

Once a small set of training or seed phrases have been provided by anadministrator, a machine learning module (e.g., an AI engine) uses thosephrases to automatically train and generate many other similar phrasesthat may be associated with the intent. This automatic training processby which a large number of similar queries are generated and associatedwith a specific intent is referred to as “fanning” and the newlygenerated queries are referred to as “fanned queries”. The machinelearning module stores the complete set of training and derived phrases(hereinafter “intent phrases”) with the intent for use duringcollaboration sessions. Subsequently as a user uses the system andutters a phrase that is similar to but not an exact match for one of theintent phrases, the intent matching module recognizes the user’s intentdespite the imperfect match and responds accordingly. In addition, whenan utterance is similar to but not exactly the same as one of the intentphrases, the system automatically saves the utterance as an additionalintent phrase associated with the intent and may train additional otherintent phrases based thereon so that the intention matching modulebecomes more intelligent over time.

In most cases a system user’s intent alone is insufficiently detailed toidentify specific information the user is seeking or how to respond andthe user has to utter or provide additional query parameters. DIALOGFLOWenables an administrator to specify a set of parameter types to extractfrom received voice messages. For example, some parameters may include adate, a time, an age, an ailment, a condition, a medication, atreatment, a procedure, a physical condition, a mental condition, etc.For each parameter type, the administrator specifies exemplary parameterphrases or data combinations (hereinafter “parameter phrases”) that asystem user may utter to indicate the parameter and, again, the machinelearning module uses the administrator specified parameter phrases totrain a larger set of parameter phrases usable for recognizing instancesof the parameter. During a collaboration session when a user query isreceived, after module 72 identifies intent, extraction module 76 usesthe parameter phrases to extract parameter values from the user’s voicemessage and the intent and extracted parameters together provide the rawmaterial needed by data operation module 64 to formulate a dataoperation to perform on the database 18 data (see again FIG. 1 ).

DIALOGFLOW allows an administrator to tag some parameters as requiredand to define feedback prompts to be presented to a user when a receivedvoice message does not include a required parameter. Thus, for instance,if a specific intent requires a date and a query associated with thatintent does not include a date parameter, the system may automaticallypresent a feedback prompt to the user requesting a date (e.g., “Whatdate range are you interested in?”).

DIALOGFLOW also guides the administrator to define intent responses. Anintent response typically includes a text response that specifies one ormore phrases, a data response or a formatted combination of text anddata that can be used to respond to a user’s query. For example, wherethe intent is to return a number of patients that meet qualifyingparameters, a response phrase may be “The number of patients that haveis .”, where the blanks represent data fields to be filled in withparameters from the voice message, data from the database, data derivedfrom the database or options specified in conjunction with the responsephrase.

Hereafter an intent and all of the information (e.g., parameters, fannedqueries, data operations and answer phrases) related to the specificintent that is specified by the system will be referred to as an intentand supporting information at times in the interest of simplifying thisexplanation.

In at least some cases, the module 72 can identify at least one intentwith a query. In at least some cases, the query can be an audible query.In at least some cases, the at least one intent can be an intent relatedto a clinical trial. In at least some cases, the at least one intent canbe related to a drug. In at least some cases, the intent can be referredto as a drug intent if the intent is related to a drug. In at least somecases, the drug intent can be related to a drug such as chemotherapy. Inat least some cases, the drug intent can be an intent related to a PARPinhibitor intent. In at least some cases, the at least one intent can berelated to a gene. In at least some cases, the at least one intent canbe related to immunology. In at least some cases, the at least oneintent can be related to a knowledge database. In at least some cases,the at least one intent can be related to testing methods. In at leastsome cases, the at least one intent can be related to a gene panel. Inat least some cases, the at least one intent can be related to a report.In at least some cases, the at least one intent can be related to anorganoid process. In at least some cases, the at least one intent can berelated to imaging. In at least some cases, the at least one intent canbe related to a pathogen. In at least some cases, the at least oneintent can be related to a vaccine.

In FIG. 1 , response module 76 uses the response phrases to generateresponses and, more specifically, audio response files that are providedback to collaboration server 12. Again, it is contemplated that atypical system may include hundreds or even thousands of responsephrases, at least one response phrase format or structure for eachintent supported by the system.

In the illustrated exemplary system 10, AI server 14 does not controldatabase 18 and therefore transmits the intent and extracted parametersback to collaboration server 12 which runs data operation module 64. Inthe present case it is contemplated that many data responses may not beable to be presented to a user in an easily digestible audio responsefile. For instance, in some cases a data response may include agraphical presentation of comparative cancer data which simply cannot beaudibly described in a way that is easy to aurally comprehend. In thesecases, after data operation module 64 receives a data response fromdatabase 18, module 64 may pass that data on to visual response module62 which generates a suitable visual response to the user’s query whichin turn transmits the visual response via transceiver 16 to device 20for presentation.

In at least some cases summary audio responses may be formulated by thesystem 10 where appropriate and broadcast via device 20. For instance,in some cases a data response may simply include a list type subset ofdatabase data that is to form the basis for additional searching anddata manipulation. For example, a sub-dataset may include data for allmale cancer patients since 1998 that have had an adverse reaction totaking any medication. This sub-dataset may operate as data for asubsequent query limiting the cancer type to pancreatic or the treatmentto treatment XXX or any other more detailed combination of parameters.In these cases where a database subset is limited, an appropriate audioresponse file may include a summary response such as, for instance, “Asubset of data for all male cancer patients that have had an adversereaction to taking any medication has been identified.” (See 66 in FIG.1 .) This response phrase would be specified via the DIALOGFLOW or otherconversation defining software applications.

In at least some cases it is contemplated that the system may not beable to associate an oncologist’s voice query (i.e., an audible query)with an intent or system supported parameters with a high level ofconfidence. In some cases it is contemplated that the AI server 14 maybe able to assign confidence factors to each intent and extractedparameters and may be programmed to pose one or more probing queriesback to an oncologist when intent or a parameter value confidence factoris below some threshold level. In some cases the probing feedback querymay be tailored or customized to known structure or data content withinthe database 18 or intents and parameters supported by AI server 14 tohelp steer the oncologist toward system supported queries.

In cases where an intent and/or extracted parameters are not supportedby the AI server or other system processes, it is contemplated thatsystem 10 will generate a record of the unsupported queries forconsideration by an administrator as well as for subsequent access bythe oncologist. In these cases it is contemplated that the system willpresent unsupported queries and related information to an administratorduring a system maintenance session so that the administrator candetermine if new intents and/or parameters should be specified inDIALOGFLOW or via some other query flow application. In a case where anadministrator specifies a new intent and/or parameters, the system mayupdate the collaboration record including the unsupported query toprovide a data response to the query and to indicate that the query willnow be supported and the oncologist may be notified via e-mail, text, orin some other fashion that the query will be supported during subsequentcollaboration sessions.

In some cases, the database 18 may include an electronic health recorddatabase from a hospital or a hospital system. In other cases, thedatabase 18 may include an electronic data warehouse with data that hasbeen extracted from an EHR, transformed, and loaded into amultidimensional data format. In other cases, the database 18 mayinclude data that has been collected from multiple hospitals, clinics,health systems, and other providers, either across the United Statesand/or internationally. The data in database 18 may include clinicaldata elements that reflect the health condition over time of multiplepatients. Clinical data elements may include, but are not limited to,Demographics, Age/DOB, Gender, Race/Ethnicity, Institution, RelevantComorbidities, Smoking History, Diagnosis, Site (Tissue of Origin), Dateof Initial Diagnosis, Histology, Histologic Grade, Metastatic Diagnosis,Date of Metastatic Diagnosis, Site(s) of Metastasis, Stage (e.g., TNM,ISS, DSS, FAB, RAI, Binet), Assessments, Labs & Molecular Pathology,Type of Lab (e.g. CBS, CMP, PSA, CEA), Lab Results and Units, Date ofLab, Performance Status (e.g. ECOG, Karnofsky), Performance StatusScore, Date of Performance Status, Date of Molecular Pathology Test,Gene/Biomarker/Assay, Gene/Biomarker/Assay Result (e.g. Positive,Negative, Equivocal, Mutated, Wild Type), Molecular Pathology Method(e.g., IHC, FISH, NGS), Molecular Pathology Provider, AdditionalSubtype-specific data elements (e.g. PSA for Prostate), Treatment, DrugName, Drug Start Date, Drug End Date, Drug Dosage and Units, Drug Numberof Cycles, Surgical Procedure Type, Date of Surgical Procedure,Radiation Site, Radiation Modality, Radiation Start Date, Radiation EndDate, Radiation Total Dose Delivered, Radiation Total FractionsDelivered, Outcomes, Response to Therapy (e.g. CR, PR, SD, PD), RECIST,Date of Outcome / Observation, Date of Progression, Date of Recurrence,Adverse Event to Therapy, Adverse Event Date of Presentation, AdverseEvent Grade, Date of Death, Date of Last Follow-up, and Disease Statusat Last Follow Up. The information in database 18 may have data in astructured form, for instance through the use of a data dictionary ormetadata repository, which is a repository of information about theinformation such as meaning, relationships to other data, origin, usage,and format. The information in database 18 may be in the form oforiginal medical records, such as pathology reports, progress notes,DICOM images, medication lists, and the like.

The database 18 may further include other health data associated witheach patient, such as next-generation sequencing (NGS) informationgenerated from a patient’s blood, saliva, or other normal specimen; NGSinformation generated from a patient’s tumor specimen; imaginginformation, such as radiology images, pathology images, or extractedfeatures thereof; other -omics information, such as metabolicinformation, epigenetic analysis, proteomics information, and so forth.Examples of NGS information may include DNA sequencing information andRNA sequencing information. Examples of imaging information may includeradiotherapy imaging, such as planning CT, contours (rtstruct),radiation plan, dose distribution, cone beam CT, radiology, CTs, PETsand the like. The information in database 18 may include longitudinalinformation for patients, such as information about their medical stateat the time of a diagnosis (such as a cancer diagnosis), six month afterdiagnosis, one year after diagnosis, eighteen months after diagnosis,two years after diagnosis, thirty months after diagnosis, three yearsafter diagnosis, forty-two months after diagnosis, four years afterdiagnosis, and so forth. The information in database 18 may includeprotected health information. The information in database 18 may includeinformation that has been de-identified. For instance, the informationin database 18 may be in a structured format which does not include (1)patient names; (2) all geographic subdivisions smaller than a state,including street address, city, county, precinct, ZIP code, and theirequivalent geocodes, except for the initial three digits of the ZIP codeif, according to the current publicly available data from the Bureau ofthe Census: (a) The geographic unit formed by combining all ZIP codeswith the same three initial digits contains more than 20,000 people; and(b) The initial three digits of a ZIP code for all such geographic unitscontaining 20,000 or fewer people is changed to 000; (3) All elements ofdates (except year) for dates that are directly related to anindividual, including birth date, admission date, discharge date, deathdate, and all ages over 89 and all elements of dates (including year)indicative of such age, except that such ages and elements may beaggregated into a single category of age 90 or older; (4) Telephonenumbers; (5) Vehicle identifiers and serial numbers, including licenseplate numbers; (6) Fax numbers; (7) Device identifiers and serialnumbers; (8) Email addresses; (9) Web Universal Resource Locators(URLs); (10) Social security numbers; (11) Internet Protocol (IP)addresses; (12) Medical record numbers; (13) Biometric identifiers,including finger and voice prints; (14) Health plan beneficiary numbers;(15) Full-face photographs and any comparable images; (16) Accountnumbers; (17) Certificate/license numbers; and (18) Any other uniqueidentifying number, characteristic, or code. The number of records ofinformation in the database 18 may reflect information from 10, 100,1,000, 10,000, 100,000, 1,000,000, 10,000,000 or more patients. Otherexamples of the type of information in database 18 are described in U.S.Pat. Publication No. 2021/0090694, published Mar. 25, 2021, the contentsof which are incorporated herein by reference in their entirety, for allpurposes.

The collaboration device 20 can reduce the amount of private healthinformation that may currently be included in emails sent tooncologists. The collaboration device 20 can delete any generatedresponses (visual or audio) from the memory 34 after the response isoutput through the speakers 44. Thus, the private health information maybe eliminated from memories outside the database 18. The collaborationdevice 20 can be configured to recognize a “Can you repeat that” commandthat causes the collaboration device 20 to re-query the last questionthat the oncologist asked (e.g., from the collaboration server 12) andonce again play back the response and remove it from the memory 34.

Referring to FIG. 2 as well as FIG. 3 , an embodiment of an exemplarysecond collaboration device 20 a is shown. The collaboration device 20 acan include a second external housing 22 a that includes sevensubstantially flat faces. The collaboration device can include a secondlight device 50 a, one or more second microphones 38 a that can bepositioned near small circular openings in the second external housing22 a, and second speakers 44 a that can be positioned near substantiallyoval-shaped openings in the second external housing 22 a. In someembodiments, the second external housing 22 a can be metallic, and mayinclude stainless steel and/or anodized steel. The second collaborationdevice 20 a can be about one and a half inches wide, one and a halfinches tall, and one and a half inches deep. In some embodiments, thesecond speakers 44 a can be headphones coupled to the secondcollaboration device wirelessly (e.g., via BLUETOOTH) or via a wiredconnection (e.g., 3.5 mm jack audio cable.

Referring now to FIGS. 2 and 3 as well as FIG. 4 , the secondcollaboration device 20 a can include the processor 30 linked to each ofbattery 32, memory 34, transceiver 36, input buttons 52, display screens48, and sensors 40, 42 as in the collaboration device 20 describedabove. The processor 30 can also be linked to the second light device 50a, the second microphones 38 a, and the second speakers 44 a. Thetransceiver 36 may be configured to communicate using a 5G cellularnetwork protocol.

The second collaboration device 20 a can include a touch interface 23that can receive inputs from the oncologist. The touch interface 23 canbe linked to the processor 30. The touch interface 23 can include secondexternal housing 22 a and sensors (not shown) such as force sensorscoupled to the second external housing 22 a. The sensors can sensedeflections of the second external housing 22 a and output acorresponding signal to the processor 30. In some embodiments, thesensors can include New Degree Technology force sensor film. The secondexternal housing 22 a can be marked (e.g., engraved) at appropriatelocations to identify different sensing areas that may correspond todifferent virtual buttons (e.g., “power,” “ok,” “mute,” etc.) to theoncologist.

In some embodiments, the touch interface 23 can include one or moretouch sensors arranged to receive inputs from the oncologist. The one ormore touch sensors may include one or more capacitive touch sensorsconfigured to output signals to the processor 30. In some embodiments,the touch sensors can be positioned under the one or more displayscreens 48. The processor 30 can prompt the user to provide inputs(e.g., by displaying instructions and/or prompts on the one or moredisplay screens 48 and/or emitting audible instructions at the speakers44A) and receive signals associated with the oncologist from the one ormore touch sensors. The processor 30 may authenticate the oncologistbased on the signals by determining if the signals match a predeterminedfingerprint profile associated with the oncologist. The processor 30 maydetermine a selection (e.g., a menu option, a power on/off command, amute command, etc.) based on the signals.

The second collaboration device 20 a can also include a power supplyinterface module 33 coupled to the battery 32 in order to supply powerto the battery 32. The power supply interface 33 can include anyappropriate hardware for regulating the power supplied to the battery.The power supply interface module 33 can include a hardwired interface(not shown) for connecting to a complementary interface coupled to anexternal power source. The hardwired interface can include copper orgold contact pins and may be magnetic. Alternatively, the power supplyinterface module 33 can include one or more transformers (not shown) forreceiving power wirelessly. The one or more transformers may include apot core transformer configured to receive power transmitted atapproximately 300 MHz, higher than other wireless charging systems thatuse standards such as the Qi wireless power transfer standard. Certainwireless charging systems may use coils to implement wireless charging.These wireless charging systems, which may be compatible with the Qistandard, may not be used in the second collaboration device 20 a due tothe small size of the collaboration device 20 a. In embodiments wherethe second external housing 22 a is metallic, care must be taken toensure the wireless charging does not result in the second externalhousing 22 a heating up. The pot core transformer may funnel transmittedenergy to the battery 32 to prevent energy from being dispersed in thesecond external housing 22 a better than coil transformers.

The second collaboration device 20 a can include a mesh networkingtransceiver 37 linked to the processor 30. The mesh networkingtransceiver 37 may be a WiFi transceiver, a Z-WAVE transceiver, a ZIGBEEtransceiver, a combination of different types of transceivers (e.g.,including Z-WAVE and ZIGBEE), etc. In particular, the mesh networkingtransceiver 37 can communicate on a frequency other than one or more ofthe frequencies used by the transceiver 36 to communicate with thetransceiver 16 as shown in FIG. 1 .

While the transceiver 36 can be used to communicate with transceiversincluded in other collaboration devices, the mesh networking transceiver37 can reduce potential transmission traffic on the communicationfrequency used by the transceiver 36 to communicate with the transceiver16. For example, the transceiver 36 can be used to communicate with thetransceiver 16 on a 2.4 GHz frequency (e.g., using a WiFi or BLUETOOTHprotocol) and the mesh networking transceiver 37 can be used tocommunicate with another mesh networking transceiver 37 located inanother collaboration device 20 a on a 900 MHz frequency (e.g., using aZ-Wave protocol).

The transceiver 36 and/or the mesh networking transceiver 37 can beconfigured to transmit and receive information using ultra wideband(UWB) protocol. UWB may be useful for detecting a real-time location ofthe second collaboration device 20 a and/or tracking how the oncologistis operating the second collaboration device 20 a.

The second collaboration device 20 a can include a secure element 35linked to the processor 30. The secure element 35 can performauthentication and encryption tasks such as key storage. The secureelement can include an ATECC608A microchip from Microchip TechnologyInc.

It is understood that at least a portion of the components included inthe second collaboration device 20 a, such as the touch interface 23,second light device 50 a, the second microphones 38 a, and the secondspeakers 44 a, the mesh networking transceiver 37, the secure element35, and the power supply interface module 33, can be included in thecollaboration device 20, and may be linked to the processor 30 and/orbattery 32. In some embodiments, the processor 30 can be a SoC such asthe QUALCOMM QCS405 SoC described above. The SoC can be used toimplement edge computing as well as machine learning processes locallyin the second collaboration device.

Referring now to FIG. 5 , a process 100 for facilitating a collaborativesession that is consistent with at least some aspects of the presentdisclosure and that may be implemented via the FIGS. 1 and 2 system isillustrated. Process 100 will initially be described in the context of asystem where the only interface device used by an oncologist is thecollaboration device 20 (e.g., the system does not include asupplemental or additional large display screen or other emissivesurface for presenting additional visual data response representationsto a user). In this type of system the portions of process 100 shownsurrounded by dashed lines would not be present.

Referring to FIGS. 1, 2, and 5 , at process block 102 an industryspecific dataset is stored and maintained in database 18. At block 104,the intent matching, parameter extracting and audio response modules 72,74 and 76, respectively, are trained using DIALOGFLOW or some otherconversation defining application as described above. In addition, atblock 104 the visual response module 62 is programmed to receive dataresponses from module 64 where the responses provide seed data forconfiguring graphical or other visual representations of the responseinformation.

Referring still to FIGS. 1 and 5 , in a system only including interfacedevice 20, control passes from block 104 to block 106 wherecollaboration device 20 monitors for activation (e.g., voice activation,movement, selection of an activation button, etc.). Optionally, betweenthose steps, the collaboration device may be paired with a proximatedisplay device, as at block 105. Once collaboration device 20 isactivated at block 108, control passes to block 112 where a voice signalis captured by device 20 and the voice signal is transmitted 57 tocollaboration server 12. At block 114, the captured voice signal istransmitted 61 to AI server 14 where ASR module 70 transcribes the voicesignal to text, intent matching module 72 examines the text file todetermine the oncologist’s intent, and parameter extraction module 74extracts key parameter values from the transcribed text. The text file,intent and extracted parameters are passed back 63 to collaborationserver 12 and more specifically to data operation module 64.

At block 116, data operation module 64 instantiates a new collaborationrecord on database 18 and stores 65 the text file in the collaborationrecord. Operation module 64 also uses the intent and extractedparameters and associated values to construct a data operation at block118 and that operation is performed at block 120 which yields a dataresponse. At process block 124, operation module 64 provides 69 the dataresponse to AI audio response module 76 which in turn generates an audioresponse file. The audio response file is sent back 71 to collaborationapplication and sent 73, 81 to collaboration device 20 at process block126. The audio response file and related text is stored at block 126 aspart of the collaboration record. The audio response file is broadcast66 via device 20 speakers 44 for the oncologist to hear at block 128after which control passes back up to block 106 wherein the processcontinues to cycle indefinitely.

After the data operation is performed at block 120, the process 100 mayinclude an optional subprocess 130 in which the data response isevaluated to determine if it includes a visual response, as at block132, in addition to the audio response described above. If not, thesubprocess may end. If so, and similar to the processes described abovefor the audio response, the operation module formulates a suitablevisual response file at process block 134 and then transmits that visualresponse file to a paired display to present to the user at processblock 136.

Where a collaboration session persists for multiple rounds of oncologistqueries and system responses, each of an oncologist’s voice message andassociated text and response file and associated text is stored in thecollaboration record so that a series of back and forth voice andresponse messages are captured for subsequent access and consideration.

In at least some embodiments the system also supports a visual outputcapability in addition to the audio file broadcasting capability toimpart process status or state information as well as at least somelevel of response data in response to user queries. For instance, inFIG. 1 , as an oncologist’s voice signal is captured by device 20 and AIserver 14 generates transcribed text, server 12 may transmit that textfile back to device 20 to be presented in real time via display 48 as afeedback mechanism so that an oncologist can ensure that the query wasaccurately perceived. Here, in some cases, the feedback text may persistuntil replaced by a visual data response where appropriate (e.g.,persists for a few seconds in most scenarios) or may persist for a setduration (e.g., 5-7 seconds). In other cases the feedback text may onlybe replaced via a next feedback text phrase so that the oncologist hasmore time to assess accuracy of the perceived utterance.

As another instance, referring still to FIG. 1 , where a data responseis suitable for visual representation or even optimal if visuallyrepresented via device display 48, the data response or a portionthereof may be provided to visual response module 62 as shown at 63. Inthese cases, module 62 uses the data response to create a visualresponse file which is transmitted (see 77 and 81) to device 20 to drivedisplay 48. In some cases the visual response presented may include atextual representation of the audio response file. In other cases thevisual response may include reminders, alerts, notifications or otheruser instructions of any type. Where visual files are generated andpresented to a user, collaboration server 12 may store all visualrepresentations as part of the ongoing collaboration record forsubsequent access.

Referring now to FIG. 6 , an exemplary collaboration conversationbetween an oncologist 150 and collaboration device 20 is illustratedwhere oncologist voice messages are shown in a left hand column 160 andinterleaved audio responses broadcast by device 20 are shown in a righthand column 162. Once device 20 is activated, device 20 responds withthe phrase “How can I help you?” to prompt the oncologist 150 toenunciate a first substantive query of the database 18. Oncologist 150responds with a first query to “Select patients with pancreatic cancer.”Here, consistent with the description above, AI server 14 (FIG. 1 )identifies intent and query parameters that are used to construct a dataoperation which yields a data response and ultimately the audio response“Patients with pancreatic cancer cohort identified.” Oncologist 150 thenenunciates a second query “Limit cohort to men.” causing the system toconstruct and perform another data operation to yield another audibleresponse. This back and forth “conversation” continues until oncologist150 ends the session.

In cases where collaboration application 60 stores collaboration recordson database 18, the system will enable an oncologist to access thoserecords subsequently to refresh memory, initiate a more detailed line ofquery aided by additional output affordances such as a large workstationdisplay screen, etc. To this end, see FIG. 7 that shows input and outputdevices at a workstation inducing a large flat panel display screen 170,a keyboard 172 and a mouse input device 174. Mouse 174 controls an onscreen pointing icon 176 for selecting on screen virtual icons and toolsas well known in the interface arts. A screen shot on display 170 showsa collaborator window 180 that includes a list of oncologist-systemcollaborations for a specific oncologist that are selectable to accesscomplete collaboration records. The list includes two columns includinga date column 182 indicating the date of a corresponding collaborationsession and a collaboration column 184 that includes a first querycorresponding toe each collaboration represented in the list. A firstentry in column 184 corresponds to the collaboration session illustratedin FIG. 6 and is shown selected via icon 176 and highlighted to indicateselection.

When the first entry in column 184 is selected, the screen shot 190shown in FIG. 8 may be presented that includes the full collaborationrecord in text with oncologist queries in a first column 192 and theaudio system responses represented as text in a second column 194. Theexample in FIG. 8 corresponds to the conversation in FIG. 6 . Here,while the conversation is presented as text, it is contemplated that theoncologist may play an audio recording of the conversation back as amemory aid and to that end, a “Play” icon 196 is provided that isselectable to replay collaboration audio.

While collaboration device 20 is advantageous because of its relativelysmall size and portability, in at least some cases data responsepresentation is either more suitable via visual representations thanaudio or audio representations would optimally be supplemented viavisual representations on a scale larger than afforded by device display20. To this end, it is contemplated that portable collaboration device20 may be supplemented as an output device via a proximate large flatpanel display screen when a larger visual representation of responsedata is optimal. Referring now to FIG. 9 , an input/output configuration200 that may be substituted for the collaboration device 20 in FIG. 1 isillustrated. In FIG. 9 , the input/output configuration includes aportable collaboration device 20, a proximate large flat panel displayscreen 202 and input keyboard and mouse devices 204 and 206,respectively.

Referring still to FIG. 9 , in at least some cases device 20 may beprogrammed to wirelessly “pair” with any BLUETOOTH or other wirelessprotocol enabled display screen that is in the general vicinity ofdevice 20 when some pairing event occurs. Here, a pairing event maysimply include any time device 20 is proximate a pairable display 202regardless of whether or not device 20 has been activated to listen fora user’s voice signal. In other cases, device 20 may only pair withdisplay once device 20 becomes active (e.g., the pairing event would beactivation of device 20). In still other cases, pairing may only occuronce device 20 receives a video response file that requires largedisplay 202 for content presentation (e.g., the pairing event would bereception of a video file including data optimally presented on a largedisplay screen).

Regardless of the pairing event, pairing may be automatic uponoccurrence of the event or may require some affirmative activity by theuser to pair. For instance, affirmative activity may include device 20broadcasting a voice query to the user requesting authorization to pairwith display 202 and a user voicing a “Yes” response in return.

Once device 20 is paired with display 202, an application program run bya display processor may take over the entire display desktop image andpresent a large collaboration interface via the entire display screen.In an alternative, the application may open a collaborator window 210 asshown in FIG. 9 in which to present visual response files. In FIG. 9 ,an exemplary visual response representation is shown at 212.

In at least some cases a collaborator window 210 or desktop image may bepresented automatically via display 202 when a pairing event occurs. Inother cases, even if device 20 pairs with a display 202, collaborationwindow 210 may not be provided until some secondary triggering eventoccurs like, for instance, device 20 is activated or a visual responsefile to be displayed on display 202 is received. In still other caseswindow 210 may only be presented after a user takes affirmative actionto pair device 20 and display 202.

In at least some embodiments, even when device 20 is paired with display202, response files may only be presented to a user via device 20 attimes. For instance, in many cases collaboration server 12 will onlygenerate an audio response file and in that case the audio file wouldonly be broadcast via device 20 with no visual representation on display202. Here, some user queries may result in response via only device 20,other queries may result in response via only display 152022 and stillother queries may result in combined responses via each of device 20 anddisplay 202.

As described above, in at least some embodiments all collaborationsystem communication with display 202 may be through device 20 so thatserver 12 does not communicate directly with display 202. In other casesit is contemplated that display 202 will have its own Internet of Things(IoT) address and therefore that server 12 could communicate visualresponse files directly to display 202. In this case, pairing wouldrequire location based association of device 20 and display 202 andstoring that association information in a database by server 12 so thataudio and visual response file transmission to device 20 and display 202can be coordinated.

In at least some cases it is contemplated that when a visual responsefile is presented on a paired large display 202, a coordinated visualresponse may be presented via collaboration device display 48 thatrefers the oncologist to the larger display 202. Similarly, an audiobroadcast by device 20 may direct the oncologist to the larger display202 or include some type of summary message related to the large display202 visual representation. In FIG. 7 , the illustrated audio broadcast220 summarizes the visual content on large display 202 and devicedisplay 48 directs the oncologist to refer to the larger paired display202 for more detailed information.

In still other cases, when response files would optimally be presentedvia a large format display while portable collaboration device 20 isremote from a large display so that it cannot pair, the system maynotify the oncologist that a better response can be obtained by pairingdevice 20 with a supplemental large display. Here the notification maybe presented via device display 48 or audibly via speakers 44. Thenotification may be in addition to broadcasting an audio response filewith abbreviated response data.

When system 10 presents visual data via a display screen 202 during acollaboration session, in at least some embodiments all the presentedvisual files are stored in the collaboration record for subsequentaccess. To this end see, for instance, FIG. 8 where a third recordcolumn 196 include visual response data 198 that corresponds to each ofthe audio responses in column 194. Here, each visual response isaccessible to see information presented visually during an associatedcollaboration session. FIG. 10 shows one of the visual response iconsselected which causes a subwindow 230 to open up and present the visualcontent that was presented during a prior session.

In at least some cases it is contemplated that system 10 will generatedata responses suitable for generating both audio and visual responsefiles which are stored in a collaboration record without presenting anyvisual information during a collaboration. Here, during a collaborationsession all communication is via device 20 despite generation of usefulvisual response files. The visual information may then be accessedsubsequently via an interface akin to the one shown in FIGS. 8 and 10 .

Referring now to FIG. 11 , a second exemplary system 300 that isconsistent with at least some aspects of the present disclosure isillustrated. Here, unlike the FIG. 1 system where AI processes areperformed by an independent AI server 14, the AI processes are performedby portable collaboration device 20 which passes information on tocollaboration server 12 for fulfillment or performance of dataoperations. As illustrated, the ASR, intent matching and parameterextraction modules 70, 72 and 74, respectively, are all included indevice 20. An oncologist’s voice signal captured by device 20 isprovided 310 to ASR engine 70 which generates test provided to intentmatching module 72. Module 72 identifies the oncologist’s intent andthen module 74 extracts parameters from the voice signal and each of thetext, intent and extracted parameters is wirelessly transmitted 302 viatransceiver 16 to collaboration server 12. Server 12 operates in thesame manner described above to create and build a collaboration recordbased on oncologist voice messages and system responses and also to usethe intent and parameters to formulate data operations to be performedon database 18 to generate data needed to answer oncologist queries. Thedata responses are transmitted 304 back to device 20 where audioresponse module 76 generates an audio file to drive speakers 44 andpresent the audio response.

Referring now to FIG. 12 , a third exemplary system 320 that isconsistent with at least some aspects of the present disclosure isillustrated. Similar to the second exemplary system 300 shown in FIG. 11, the AI processes performed by the independent AI server 14 areperformed by portable collaboration device 20. Unlike the secondexemplary system 300, the processes performed by the independentcollaboration server 12 are performed by portable collaboration device20. The collaboration device is linked to the database 18 in order tosend data operations 322 to the database 320 and receive data response324 from the database 18. The collaboration device 20, and morespecifically the audio response module 76, then generates an audio fileto drive speakers 44 and present the audio response as described above.

Referring to both FIGS. 9 and 12 , having at least a portion ofprocesses performed by the AI provider server 14 and the collaborationserver 12 implemented locally in the collaboration device 20 can reducelatency of generating audio response by up to two seconds. In someembodiments, at least a portion of the modules 62, 64, 70, 72, 74, 76and/or the collaborator application 60 can be stored in thecollaboration server 12 or the AI provider server 14 and periodicallyupdated and pushed to the collaboration device 20. In other words, thecollaboration server 12 or the AI provider server 14 can store the mostcurrent versions of the modules 62, 64, 70, 72, 74, 76 and/or thecollaborator application 60 and periodically (e.g., once per day orweek) update the processes stored on the collaboration device 20 toinclude the processes of the most current modules 62, 64, 70, 72, 74, 76and/or the collaborator application 60 stored on the collaborationserver 12 or the AI provider server 14. In this way, the processesexecuted by the collaboration device 20 can be continually updated whilereducing the latency of generating audio responses based on input fromthe oncologist.

Referring to FIG. 13 , a number of collaboration devices 20 b-e cancommunicate using a mesh networking technique with each other and/orwith at least one of a first transceiver 16 a and a second transceiver16 b. The number of collaboration devices 20 b-e can include a thirdexemplary collaboration device 20 b, a fourth exemplary collaborationdevice 20 c, a fifth exemplary collaboration device 20 d, and a sixthexemplary collaboration device 20 b. Each of the collaboration devices20 b-e can include at least a portion of the components of thecollaboration device 20 or the second collaboration device 20 adescribed above. In some embodiments, each of the collaboration devices20 b-e can be the collaboration device 20 or the second collaborationdevice 20 a. While four collaboration devices 20 b-e are shown, it isunderstood that more than four collaboration devices can be used. Eachof the first transceiver 16 a and the second transceiver 16 b can be thetransceiver 16 described above.

The number of collaboration devices 20 b-e can each include acorresponding transceiver 36 b-e, each of which can be substantially thesame as the transceiver 36 described above. Each of the numbercollaboration devices 20 b-e may be linked to the first transceiver 16 aand/or the second transceiver 16 b in order to transmit voice andmessage signals to the first transceiver 16 a and/or the secondtransceiver 16 b using the corresponding transceiver 36 b-e included inone of the collaboration devices 20 b-e. For example, the fourthcollaboration device 20 c can be linked to the second transceiver 16 b,the fifth collaboration device 20 d can be linked to the firsttransceiver 16 a and the second transceiver 16 b, and the sixthcollaboration device 20 e can be linked to the second transceiver 16 b.

The first transceiver 16 a and the second transceiver 16 b can be linkedto the collaboration server 12 to transmit voice signal messages to thecollaboration server 12 and receive visual response files and audioresponse files transmitted from the collaboration server 12 as describedabove. The collaboration server 12 can be linked to the AI providerserver 14 to transmit voice signal messages and data responses to the AIprovider server 14 and receive the text files, the matching intent, andthe extracted parameters associated with the voice signal messages aswell as the audio response files associated with data responsestransmitted from the AI provider server 14. The collaboration server 12can be linked to the database 18 in order to create collaborationrecords and perform data operations in the database 18, as well asreceive data response transmitted from the database 18.

Each of the number of collaboration devices 20 b-e can communicatedirectly with at least one other collaboration devices 20 b-e to form amesh network. The number of collaboration devices 20 b-e can communicatewith each other using a communication protocol supported by thecorresponding transceivers 36 b-e, for example WiFi protocol, UWBprotocol, and/or ZIGBEE protocol. Each of the number of collaborationdevices 20 b-e can also include a corresponding mesh networkingtransceiver 37 b-e, each of which can be substantially the same as themesh networking transceiver 37 described above.

The number of collaboration devices 20 b-e can communicate directly witheach other using the corresponding mesh networking transceivers 37 b-e,which may reduce transmission traffic on the communication frequencyused by the corresponding transceivers 36 b-e. The direct connectivitybetween the collaboration devices 20 b-e can be helpful if one of thenumber of collaboration devices 20 b-e cannot communicate with any ofthe first transceiver 16 a or the second transceiver 16 b. For example,if the third collaboration device 20 b cannot communicate with thetransceivers 16 a-b, the third collaboration device 20 b can routecommunications through the fifth collaboration device 20 d that islinked to the first transceiver 16 a in order to transmit voice signalmessages and receive transmitted audio and/or visual response files asdescribed herein. Thus, all of the collaboration devices 20 b-e can belinked to the collaboration server 12.

The location of each of the number of collaboration devices 20 b-e canbe determined in order to potentially prevent loss or theft of thenumber of collaboration devices 20 b-e. A monitoring process that may beincluded on a server in communication with the first transceiver 16 aand the second transceiver 16 b (e.g., the collaboration server 12) canmonitor the location of the collaboration devices 20 b-e. The monitoringprocess can cause heartbeat messages to be transmitted from thetransceivers 16 a-b to the collaboration devices 20 b-e and can receiveheartbeat messages transmitted from the collaboration devices 20 b-e tothe transceivers 16 a-b. The monitoring process can then determine thelocation of each of the collaboration devices 20 b-e based on theheartbeat messages.

In some embodiments, each of the number of collaboration devices 20 b-ecan transmit heartbeat messages at predetermined intervals (e.g., everyten minutes) to the first transceiver 16 a and the second transceiver 16b, which may retransmit the heartbeat messages to another device such asthe collaboration server 12. In this way, other devices and/or processessuch as the collaboration server 12 can track and/or triangulate thelocation of a given collaboration device (e.g., the fifth collaborationdevice 20 d). In some embodiments, the first transceiver 16 a and thesecond transceiver 16 b can be associated with wireless access point MACaddresses that may be associated with GPS coordinates. The monitoringprocess can then estimate the location of the collaboration devices 20b-e based on the GPS coordinates, which are (indirectly) associated withthe transceivers 16 a-b. The monitoring process can determine that agiven collaboration device (e.g., the third collaboration device 20 e)transmitted heartbeat messages to both transceivers 16 a-b, and estimatethe location of the given collaboration device based on the GPSlocations associated with the transceivers 16 a-b. The GPS coordinatesand/or MAC addresses can be stored in the collaboration server 12.

If the heartbeat message transmitted by one of the number ofcollaboration devices 20 be is not received by both the firsttransceiver 16 a and the second transceiver 16 b, the monitoring processmay determine that the device is lost and notify a system administratorand/or monitoring process that the device is lost. The first transceiver16 a and the second transceiver 16 b may also transmit heartbeatmessages to the number of collaboration devices 20 b-e in order toverify the device has not been potentially lost or stolen. In someembodiments, if one of the number of collaboration devices 20 b-e suchas the fifth collaboration device 20 d does not receive the heartbeatmessages from the transceivers 16 a-b at the predetermined interval, thefifth collaboration device 20 d may enter a restricted mode thatrestricts processes that can be executed by the fifth collaborationdevice 20 d and/or lock the fifth collaboration device 20 d to helpprevent potential tampering with sensitive data.

Alternatively or in addition to using the transceivers 16 a-b to tracklocations of the number of collaboration devices 20 b-e, thecollaboration devices 20 b-e themselves can be used to track each other.More specifically, the one or more collaboration devices 20 b-e cantrack another one of the collaboration devices 20 b-e using one or moreof the direct connections between collaboration devices 20 b-e. One ofthe collaboration devices 20 b-e may communicate directly with anotherone of the collaboration devices 20 b-e using UWB protocol. For example,the third collaboration device 20 b can be linked to the fifthcollaboration device 20 d and the sixth collaboration device 20 e. Thethird collaboration device 20 b can send heartbeat messages to the fifthcollaboration device 20 d and the sixth collaboration device 20 e, andin response, the fifth collaboration device 20 d and the sixthcollaboration device 20 e can send heartbeat messages back to the thirdcollaboration device 20 b. If the third collaboration device 20 b doesnot receive heartbeat messages back from the fifth collaboration device20 d and the sixth collaboration device 20 e, the third collaborationdevice 20 b may enter a restricted mode that restricts processes thatcan be executed by the third collaboration device 20 b and/or lock thethird collaboration device 20 b to help prevent potential tampering withsensitive data. Additionally, the fifth collaboration device 20 d and/orthe sixth collaboration device 20 e may send a notification that thethird collaboration device 20 b has been potentially lost or stolen toat least one of the transceivers 16 a-b. The transceivers 16 a-b maytransmit the notification(s) to the collaboration server 12 for furtherprocessing.

In some embodiments, at least a portion of the processes stored on andexecuted by the AI server 14 and/or the collaboration server 12 can bestored locally on the collaboration devices 20 b-e. In theseembodiments, the processes stored on and executed by the AI server 14and the collaboration server 12 can be continually updated, for example,by an external program or internally by the collaboration server 12and/or the AI server 14. As the processes are updated, the collaborationserver 12 and/or the AI server can update the corresponding processstored on the collaboration devices 20 b-e. The processes, which caninclude speech recognition, intent prediction, analysis, and/or routingprocesses, can be updated based on data generated by the collaborationdevices 20 b-e.

For example, the third collaboration device 20 b and the fourthcollaboration device 20 c may receive voice signal messages withdifferent phrases (e.g., phrases with different word choices) thatcorrespond to the same intent. The AI server 14 can then learn that thedifferent phrases match the same intent and update the associated module(e.g., the intent matching module 72 shown in FIG. 1 ) accordingly. Someof the collaboration devices 20 b-e can be located within the sameinstitution (e.g., the third collaboration device 20 b, the fifthcollaboration device 20 d, and the sixth collaboration device 20 e), andothers can be located in another institution (e.g., the fourthcollaboration device 20 c). In this way, the AI server 14 and/or thecollaboration server 12 can be updated based on feedback from multipleoncologists from multiple institutions.

Additionally or alternatively, the processes stored on and executed bythe AI server 14 and/or the collaboration server 12 can be updated basedon external processes. For example, an administrator can add intents tothe AI provider server 14. The AI provided server can then uploadupdated processes to the collaboration devices 20 b-e.

After an audible collaboration session, it is often difficult to getback into the same dialog flow at a later time as it is difficult toremember the back and forth communication that comprises the dialog. Forthis reason, in at least some cases a system will enable a user toreinsert herself into a flow using a display screen like the one shownin FIG. 8 . Thus, in FIG. 8 , a “Continue” button 197 is presented whichis selectable to place the overall system 10 in the state that existedat the end of the session. Here, the “state” means that all the contextassociated with the line of questioning at the end of the session isreinstated (e.g., subsets of data, qualifying parameters, etc.), so thatthe oncologist can pick up where she left off if that is desired).

One problem oncologists and doctors in general have is that they need toenter notes into patient records every time they encounter and treatpatients. At least some studies have indicated that a typical oncologistspends upwards of 1.5 hours every day memorializing events and thoughtsin patent notes. Some oncologists craft record or document notes duringpatient visits while others wait until they have a break or until theyare “off work” to craft notes. Where an oncologist crafts a note whilewith a patient, the doctor’s attention is split between the note and thepatient which is not ideal. Where an oncologist crafts a note subsequentto a patient visit, thoughts, observations and findings are oftenmisremembered or captured with less detail.

To address this problem, in at least some cases portable collaborationdevice 20 may be programmed to “listen” to an oncologist-patient careepisode and record at least portions of oncologist and patient dialogessentially in real time as a “raw transcription”. In addition, a systemprocessor may be programmed to process the raw transcription datathrough OCR and NLP algorithms to identify words, phrases and othercontent with the captured raw voice signals. In at least some cases itis contemplated that a processor may be trained using DIALOGFLOW or someother AI software program to recognize an oncologist’s intent fromcaptured words and phrases as well as various parameters needed toinstantiate different types of structured notes, records or otherdocuments that are consistent with one or more of the oncologist’sintents. In addition, it is contemplated that the processor may be ableto take into account other patient visit circumstances when discerningoncologist intent as well as identifying important parameters forspecific structured notes, records or documents.

For instance, while speaking with a patient that has pancreatic cancer,the processor may use an oncologist’s appointment schedule toautomatically identify a patient as well as to access the patient’smedical records to be used as context for voice messages captured duringa patient visit. As the oncologist and patient speak, the processor maybe programmed to discern the oncologist’s voice and the patient’s voice.Here, over time the processor would train to the oncologist’s voice andbe able to recognize the oncologist’s voice based on tone, pitch, voicequality, etc. and would be programmed to assume that other voice signalsnot fitting the oncologists belong to the patient.

In at least some cases the oncologist could intentionally indicate astructured note type for the system to generate. For instance, in asimple case, the system may be programmed to generate five differentstructured note types where each type includes a different subset of 15different parameters. Here, during DIALOGFLOW training, an administratormay provide five different phrases for each of the five different notetypes where each phrase is associated with an intent to generate anassociated note type. The processor would train on the five phrases foreach note type and come up with many other phrases to associate with thenote type intent. In addition, during training, the 15 parameter subsetsfor each note type would be specified. Moreover, a structured note typewould be created and stored in a structured note database for use ininstantiating specific instances of the note type for specific patientvisits. Furthermore, feedback queries for at least required parametersmay be developed and stored as in the case of the DIALOGFLOW systemdescribed above.

During an oncologist-patient visit, when the oncologist wants the systemto generate a specific note type, the oncologist may simply activatedevice 20 by uttering “Go One” and then a phrase like “Create aninstance of the first note type”. The processor, recognizing the intentto create an instance of the first note type then listens during thedialog to pick out required parameters to instantiate the instance ofthe note type. In at least some cases if the system cannot identify someparameter(s) required for the note instance, device 20 may be programmedto query the oncologist for the missing parameter(s). Feedback queriesmay be generated during a patient visit, immediately after the visitwhile facts and information about the visit are fresh in theoncologist’s mind or at some other scheduled time like a break, ascheduled office hour, etc.

In other cases instead of requiring a physician to voice a specific notetype to be created, the system may listen to the oncologist-patientdialog and identify an oncologist’s intent from the ongoing dialogwithout some specific request.

Any of a raw transcription, note, record or other document generated bythe system during or associated with a patient visit may be stored in apatient’s EMR or any other suitable database. The AI can learn over timefrom oncologist utterances and become smarter as described above. Inaddition, a structured note may be presented to an oncologist forconsideration prior to or after storage so that the oncologist canconfirm the information in the structured record. In cases where anoncologist changes information captured by the system, any change may beprovided back to a system processor and used to further train theprocessor AI to more effectively capture intent and/or parameters in thefuture.

In at least some cases another document type that the system mayautomatically generate is a billing document. Again, here, a systemprocessor may “listen” to what an oncologist is saying during a patientvisit and may discern an intent that has a billing ramification. At thatpoint the processor may start to listen for other parameters toinstantiate a complete billing record or document. In some cases abilling record may be automatically sent to a billing system or may bepresented in some fashion to the oncologist to confirm the accuracy ofthe billing record prior to forwarding.

In still other cases another document type the system may automaticallygenerate while listening to an oncologist is a schedule appointment.Here, again, a processor may be able to discern oncologist intent toschedule an appointment from many different utterances and may thensimply listen for other parameters needed to instantiate a completeevent scheduling action.

In particularly advantageous systems, a processor may be programmed tolisten to an oncologist and automatically identify several simultaneousintents to generate several different types of notes, records ordocuments, and may monitor oncologist utterances to identify allparameters required for each of the simultaneous intents. For instance,where the processor determines that a billable activity or event isoccurring and that an oncologist wants a structured patient visit notegenerated at the same time, where each of a structured bill and thestructured note requires a separate subset of 15 different parameters,the processor would listen to oncologist utterances for all of theparameters to instantiate each of a bill record and a patient visitnote. Again, where the system fails to capture required parameters, theprocessor may generate and broadcast or present (e.g., visually on adisplay) queries to the oncologist to fill out the required informationat an appropriate time.

In some cases it is contemplated that an oncologist may indicateautomatic document preferences for each patient visit where the systemthen automatically assumes an intent associated with each preferreddocument type and simply listens to the oncologist-patient dialog toidentify parameters required to instantiate instances of each of thepreferred document types for each patient visit. Thus, for instance, oneoncologist may want the system to generate a structured patent visitnote and a structured bill record as well as to tee up next visitscheduling options for each patient visit the oncologist participatesin. Here, at the beginning of each scheduled patient visit session, thesystem immediately identifies three intents, a patient visit noteintent, a bill record intent and a scheduling activity intent. Thesystem accesses a structured record for each of the intents and proceedsto capture all required parameters for the intents. For the schedulingactivity intent, the system may identify specific activities to bescheduled based on captured parameters and then at some appropriate time(e.g., last 5 minutes of the scheduled patient visit), may present oneor more scheduling options for the specific activity to the oncologistand patient. Here, the oncologist and patent may accept to reject anysuggested activity to schedule or the time(s) suggested for theactivity.

In still other cases, after a system processor identifies an intentbased on oncologist-patient dialog, the processor may be programmed tobroadcast a query confirming the intent. For instance, where the systemidentifies an intent to generate a patient visit note, the processor maybe programmed to broadcast the query “Would you like to have a patientvisit note generated for this visit?” Here, an affirmative responsewould cause the processor to identify a structured note format andproceed to collect note format parameters to instantiate the note.

In at least some embodiments a collaboration device 20 may listen in onall utterances by an oncologist and many oncologists may use devices 20to capture their utterances and raw voice messages. For instance, thesystem may capture all of an oncologist’s utterances during patientvisits, while participating in tumor boards, during office hours, and inother circumstances when the oncologist is discussing any aspect ofcancer care. Here, a system processor or server may be programmed torecognize all utterances by an associated oncologist and distinguishthose from utterances of others (e.g., patients, other healthcareworkers, other researchers, etc.). The processor may store all or atleast a subset of the oncologist’s raw voice messages/utterances and mayprocess those utterances to identify text, words and phrases, contextsand ultimately impressions of the oncologist. For instance, oneimpression may be that for a pancreatic cancer patient that initiallyresponded well to medication AAA where the medication is no longereffective, medication BBB should be employed as a next line of attack.

While the system may identify and automatically use discernedimpressions in some cases, in other cases the system may be programmedto immediately present perceived impressions to an oncologist and allowthe oncologist to confirm or reject the impression. Rejected impressionsmay be discarded or may be recorded to memorialize the rejection, therejection itself being an indicator of the oncologist’s impressions ingeneral and therefore useful in future analysis. Confirmed impressionswould be stored in a system database for subsequent use. In other casesimpressions may only be periodically presented to an oncologist forconfirmation or rejection.

Oncological impressions may be used as seed data for AI machine learningalgorithms so that, over time, the algorithms learn from the impressionsand populate databases with new data representing thoughts of theoncologist. The system may be programmed to associate different intentswith different thoughts and subsequently, when an oncologist voiceutterance is received, associate the utterance with the intent, identifyparameters related to the intent and then obtain the oncologist’s priorimpressions or thoughts and provide a response that is consistent withthe prior thought or impression.

In at least some cases where the system collects impressions from manydifferent oncologists, the system may combine impressions and thoughtsfrom multiple oncologists so that all oncologists that use the systemhave access to responses informed by at least a subset of theimpressions and thoughts from an entire group. Here, once the databaseof impressions evolves, when an oncologist utters a question to hercollaboration device 20, the system would again identify an intent aswell as required parameters to search the database for answers and mayidentify one or more impressions of interest to answer the question.

In at least some cases it is contemplated that the system will trackefficacy of cancer or other treatments automatically to be used as aquality metric related to oncological impressions. Here, efficacioustreatments would be assigned high confidence or other types of factorswhile low efficacy treatments based on relative efficacy of othertreatments for comparable cancer states. Then, when an oncologistqueries the system, the system would identify intent and requiredparameters to generate a structured data query and would returninformation related to only the most efficacious impressions.

In still other cases, the system may rank specific oncologists based onone or more factors and then present query responses based on or thatrepresent the impressions of only the “top” oncologists. For instance,oncologists may be ranked based on peer reputation, based on treatmentefficacy of their patients on a risk adjusted basis or using othermethods (e.g., differently weighted combinations of factors). Here,responses would be limited to data related to only top oncologists.

In still other cases it is contemplated that queries may be limited todata and impressions for only specific oncologists. For instance, afirst oncologist may desire the impression of a second specificoncologist on a specific cancer state. Here, the first oncologist maylimit a query to the second oncologist by specific name. For example,where the first oncologist has been collaborating with device 20 toaccess information related to a first patient, the first oncologist maysimply utter “What would Sue White say?”. In this case, a processorcapturing the query would recognize the intent for another oncologist’simpression, identify Sue White as a defining parameter and then accessimpressions associated with Sue White and regarding other contextualparameters previously captured and recognized by the system during priordialog (e.g., patient name, cancer state factors, etc.). The responsebroadcast or presented to the first oncologist would be limited to dataand information associated with Sue White.

In many cases, especially as a system is learning during use, the systemwill make mistakes and may return information that is not what has beenasked for. In some cases it will be clear from a response that the queryidentified by the system was not what an oncologist intended while inother cases a wrong response may not be facially recognizable from theresponse. In cases where a response is recognized as wrong reflecting aninaccurately identified query, one issue is that an oncologist has toreutter the query with better enunciation. In at least some cases it iscontemplated that if an oncologist rejects a response, the system mayautomatically attempt to identify a different query that the oncologistintended and a different suitable response. For instance if, uponhearing a response, an oncologist utters “No” or some other rejectingphrase, the system would recognize that response, formulate a differentquery based on the intent and parameters and then issue a differentresponse.

In some cases in addition to recognizing a wrong response, the responsewill be usable to comprehend an error in the query identified by thesystem that led to the wrong response. For instance, if an oncologistasks for some cancer state characteristic of Tom Green and the systemreturns a response “Tom Brown’s characteristic is XXX”, the answer isusable to identify that the perceived question was wrong. In this case,to eliminate the need for the oncologist to revoice an entire query, thesystem may be programmed to allow a partial query whereintent andparameters associated with the prior incorrectly perceived query areused along with additional information in the partial query to recognizea different data operation to be performed. Thus, in the above example,the oncologist may respond “No, I meant Tom Green.” Here the systemwould use prior query information including intent (e.g., thecharacteristic sought) as well as the new parameter “Tom Green” toaccess the characteristic for Tom Green. The idea here is that thesystem retains context during a dialog so that oncologists do not haveto continually re-voice complex queries that are misperceived by thesystem and instead can simply provide a subset of information in a nextquery selected to clear up any misperceptions.

In at least some cases, as indicated above, an answer to a query may notinclude any telltale signs that the query was misperceived by thesystem. In some cases it is contemplated that the system will beprogrammed to provide a confirmation broadcast or other message to anoncologist for each or at least a subset of queries that are uttered sothat the oncologist can confirm or reject the perceived query.Confirmation leads to a data operation while rejection would cause thesystem to either identify a different query or ask for restatement ofthe query. In still other cases an oncologist may be able to ask thesystem to broadcast the question (e.g., data operation) that the systemperceived for confirmation.

While the invention may be susceptible to various modifications andalternative forms, specific embodiments have been shown by way ofexample in the drawings and have been described in detail herein.However, it should be understood that the invention is not intended tobe limited to the particular forms disclosed. For example, while asphere shaped collaboration devices is described above, the portabledevice may take many different forms. For instance, referring to FIG. 14, a second exemplary collaboration device 20 a may include a cube shapeddevice including one or more emissive external surfaces for providingvisual content. As another instance, a third collaboration device mayinclude a tablet type device 20 b or any other portable device withcomponents suitable to perform the functions described above.

In still other cases, a portable collaboration device may be oneinterface device in a larger interface ecosystem that includes otherinterface devices where an oncologist has the ability to move seamlesslybetween system interface devices during collaborative sessions. Forinstance, an ecosystem may include other interface devices and inparticular, one or more stationary interface devices with betterinterface affordances like better microphones, larger speakercomponents, etc. In this regard, see for instance FIG. 15 , which showsanother exemplary interface device 20 c that is substantially largerthan interface device 20 and that is provided for stationary use at aworkstation 350. Exemplary interface 20 c includes a larger housingstructure that forms a cavity for receiving various components asdescribed above with respect to FIG. 2 . Here the speakers are largerand presumably would be higher quality than the speakers in device 20.In this case, device 20 c is intended to be used at its location on aworktop work surface, on a conference table in a conference room, etc.

In at least some exemplary contemplated systems, devices 20 and 20 c mayoperate in conjunction with each other where collaboration sessions canbe handed over from one of the devices 20 to the other 20 c to optimizefor given circumstances. For instance, if an oncologist is roaming whilecollaborating via device 20 and enters a space (e.g., arrives at aworkstation) that includes a better afforded stationary device 20 c,devices 20 and 20 c may wirelessly communicate to recognize each otherand to coordinate transfer of the collaboration session from device 20to device 20 c. Here, the collaboration session would continue, albeitusing the stationary device 20 c. Similarly if an oncologist is usingdevice 20 c to collaborate and gets up to leave the station, thecollaboration session may automatically or with user request orconfirmation, be switched over to device 20 so that the collaborationcan persist.

In still other cases a headphone, smart glasses with speakers and amicrophone, etc., may be used as a collaboration device in the disclosedsystem. In this regard see the exemplary headphone assembly 370 in FIG.16 that includes ear speakers 372 and a built in microphone 374.

While described in the context of a dedicated collaboration device,aspects of the present invention may also be implemented using any typeof computer interface device with microphones and speakers to enable auser-system conversation, regardless of whether or not the device isdedicated only to collaboration or not. For instance, a user’s laptopcomputer may be used as a collaboration device running a collaborationprogram, an existing voice activated smart speaker may be used as acollaboration device, etc.

While technology or new technology based tools are great when they workwell for its intended purposes, when technology or a tool does not workas expected by a user, the user often quickly becomes frustrated and, inmany cases simply dismisses the technology or tool reverting back toresources to complete various tasks. This tendency to quickly dismissimperfect new technology is exacerbated in cases where a user isextremely busy and therefore time constrained. Oncologists tend to beextremely busy people and therefore typically have little tolerance forineffective or inefficient technology and tools.

One problem with dialog systems like those described herein is that asystem that only supports a fraction of queries that oncologists maypose will more often than not fail to identify a correct intent forreceived queries. Here, in response, the system will either generate ananswer to a wrong intent, simply indicate that the system does notcurrently have an answer for the query posed. These types of imperfectanswers would cause frustration and in many cases, ultimately causeoncologists to dismiss these types of collaboration systems entirely.

In at least some embodiments it is contemplated that for a given datasetor record type, an essentially fulsome set of intents/parameters,related database queries and responses will be defined using DIALOGFLOWor some other dialog specifying software so that the system will be ableto effectively answer almost any query posed that is related to thedataset. Where new datasets, databases and record types are linked tothe system, additional intents and related information may be specifiedfor those datasets, databases and record types. For instance, in atleast some cases the system may be programmed to support hundreds ofthousands of different intents that include literally any foreseeableintent that may be intended by an oncologist. A team of systemadministrators/programmers works behind the scenes to identifyadditional possible intents and to supplement the system with newintents/parameters, related database queries and responses. Additionalintents may be based on the existing datasets and record types and/ordeveloped in response to new data types, new information and/or newoncological insights that evolve over time.

In cases where a system supports a massive number (e.g. tens or hundredsof thousands) of different intents, distinguishing one intent fromanother is complicated as the larger the number of supported intentsnaturally means that the differences between any intent and a set ofsimilar but different intents will be difficult to discern. The task ofcorrectly identifying an intent is exacerbated in a DIALOGFLOW typesystem where an AI engine using a query “fanning” process to generateand associate literally hundreds or even thousands of similar querieswith a specific intent during system training so that the possibility offanned queries for two or more different intents overlapping becomesappreciable.

At least some embodiments of the disclosed system will use one or anycombination of several techniques to discern an intended intent fromother system supported intents. A first technique is based on the systemoperating during a collaboration session to distinguish different“dialog paths” that occur during the session and information related toa specific dialog path is used to inform subsequent intents during thesame dialog path. For example, if a doctor asks device 20 to “give methe results of my patient, Dwayne Holder’s, sequencing report” and thenasks a subsequent question “what are the best clinical trial options”,the system determines that these questions are in a dialog path andanswers the clinical trial question based on the clinical trialrecommendations that have been provided on Dwayne Holder’s clinicalreport (e.g., the system recommends clinical trials on sequencingreports and the system access all the data in each of those reports). Inat least some embodiments only one dialog path is actively followed at atime. Nevertheless, in some cases the system maintains a memory cache ofpast dialog paths for an oncologist to inform future questions andanswers.

A second technique for discerning an intended intent in a system thatsupports a massive number of intents has the system creating “entities”around key concepts related to an oncologist’s query and associatedsystem response(s). For example, Drugs, Drug Regimen, Clinical Trial,Patient Name, Pharmaceutical Company, Mutation, Variant, Adverse Event,Drug Warning, Biomarker, Cancer Type, etc. are all examples of entitiessupported by an exemplary system. While a small number of entities areidentified here it should be appreciated that a typical system maysupport hundreds of different entities.

In at least some cases the system may be programmed to connect entitiesin a query or that are identified within a query path to form an entityset which is then usable to narrow down the list of potential answerswhich may be the best answers to a specific query. For instance, where aquery path is associated with patient Dwayne Holder and drug XXX, thosepatient and drug entities may form a set that limits the most likelyintents associated with subsequent queries. The system may also beprogrammed to leverages entities to evaluate whether a doctor’squestions are still part of the same dialog path or if a new question isrelated to a new topic that is associated with a new dialog path.

A third technique for discerning an intended intent in a system thatsupports a massive number of intents is referred to generally as“personalization”. Here, the idea is that many specific oncologistsroutinely follow similar dialog paths and voice similar queries withpersistent syntax and word choices and therefore, once the systemidentifies a specific oncologist’s persistent query characteristics andcorrectly associates those with specific intents, subsequent querieswith similar characteristics can be associated with the same intents,albeit qualified by different sets of query parameters.

In at least some cases the system builds real time profiles of eachoncologist or other system user based on the oncologist’s past querycharacteristics (e.g., word choice, syntax, etc.), query paths followed,prior system provided responses to those queries, oncologist responsesto the responses (e.g., does oncologist’s response indicate that thesystem answer and therefore discerned intent was correct), and overallsystem use. For example, when an oncologist logs into the system, thesystem may automatically link to a list of the patients that theoncologist has sent to a sequencing service provider, the results thatexist in those patients’ sequencing reports and the key therapies andclinical trials that have been recommended for those specific patients.These linked lists support the decision making process that the systemleverages to determine which question the oncologist is trying to ask(e.g., the oncologist’s intent). For example, if an oncologist logs inand recently met with patient named Dwayne Holder, even if the systemreceives distorted audio that, when converted to text reads like: “whatare the results for my quotient Lane Bolder,” the system may beprogrammed to recognize that this oncologist recently met with DwayneHolder, whose name is similar to Lane Bolder, and would proceed togenerate answers based on that recognition.

In particularly advantageous systems all three of the techniquesdescribed above are used either serially or in parallel or somecombination thereof to discern oncologist query intent. Thus, forinstance, the system may use entities to narrow down an oncologist’sintent when voicing a specific query, may further narrow down thepossible intent based on a current query path and then may select a mostlikely intent based on a personalization functionality associated withthe speaking oncologist.

In at least some cases it is contemplated that the system may providetools during a system training session to avoid subsequent intentconfusion. For instance, assume a system is already programmed tosupport 100,000 different intents when an administrator specifies a100,001st intent and three associated seed or training queries to drivean AI engine query fanning process. Here, during the fanning process asystem processor may be programmed to compare fanned queries for the100,001st intent to other queries that are associated with other intentsto identify duplicate queries or substantially identical queries. In atleast some cases the system may be programmed to automatically avoid acase where fanned queries for two or more intents are identical orsubstantially identical.

In other cases, when the system recognizes that first and second queriesassociated with first and second intents are substantially identical,the system may present a warning to the administrator enabling theadministrator to assess the situation and how to handle the confusingsituation. In some cases substantially identical fanned queries may meanthat the system already supports the newly specified intent in whichcase the administrator may simply forego enabling the new intent. Inother cases the administrator may select one of the prior and new intentto be associated with the query in question and in other cases theadministrator may allow the fanned query to be associated with twointents. In still other cases the administrator considering the twointents may decide that additional information is required foridentifying one or the other or both of the prior and new intents andmay further specify the factors to consider when identifying one or theother or both of those intents.

Where a query is associate with two intents, in operation when anoncologist voices the query, the system may identify both intents andgenerate a response query that is broadcast to the oncologist so thatthe oncologist can consider which intent was meant. In other cases itmay be that both intents are consistent with the oncologist’s voicedquery and therefore answers to both queries may be generated andsequentially broadcast to the oncologist for consideration.

While the goal of the collaboration system is to handle any questionthat can be answered using data in system datasets or databases, in atleast some cases despite the intent discerning techniques describedabove, the system may simply be unable to unambiguously identify oneintent and/or required parameters associated with an intent among themany intents supported by the system. For instance, in some cases it iscontemplated that the system may not be able to identify any intentassociated with a query or may identify two or more intents associatedwith a query. In these cases the system may be programmed to facilitatea triage process to hone in on a specific intent for the query. In thisregard, in at least some cases the system may be programmed to generateand broadcast a response query back to the oncologist indicating thatthe system could not determine the user’s intent and requesting that theoncologist restate the query.

In other cases where the system identifies two or more intents that maybe associated with the query, the system may broadcast a query to theoncologist like “Did you mean _______?, where the blank is filled inwith the first intent and perhaps related parameters gleaned from theinitial query. The system may ask about a second or other intents if theoncologist indicates that the first intent was not what was meant.

In cases where the system cannot discern a specific intent from a queryor follow-up answers from an oncologist, the system may automaticallybroadcast a message to the oncologist indicating that the system couldnot understand the query and indicating that a system administrator willbe considering the query and intent so that the system can be trained tohandle the oncologist’s query. Queries that cannot be associated withspecific intents are then presented to an administrator who can considerthe query in context (e.g., within a dialog path) and can eitherassociate the query with a specific system supported intent or specify anew intent and related (e.g., required and optional) parameters to beassociated with the query. Here, where a new intent is specified, theadministrator may specify a small set of additional seed queries for theintent and the system AI engine may facilitate a fanning process toagain generate hundreds of additional queries to associate with the newintent. The administrator then specifies one or more data operations onfor the new intent as well as an audible response file for generatingaudible responses for the intent. Upon publishing the new intent,parameters, data operations and response file to the system for use, ane-mail or other notification may be automatically generated and sent tothe oncologist that posed the initially unrecognizable query and, insome cases, a suitable answer to that query.

In cases where the system is able to associate a perceived query with asingle system supported intent and then performs a data operation toaccess data needed to formulate an audible answer, in at least somecases the databases and/or records searched will not yield results todrive an answer. For instance, in a case where an oncologist voices aquery about a specific patient by name and no information exists in thesystem databases for that patient, the data operation will not returnany data to answer the query. In this case, the system may be programmedto broadcast a message indicating that “There is no data in the systemfor the patient you identified.”

In other cases the system may, in addition to generating data that isdirectly responsive to a query, generate additional data (hereafter“supplemental data”) to supplement the responsive data. Supplementaldata can take essentially any type of form that can be supported by datain the system databases and may include, for instance, qualifyingstatements or phrases that apply to an associated directly applicableresponse phrase, additional data of interest, clinical trials that maybe related to the query, conclusions based on data, and data thatsupports answer statements.

Here, it is contemplated that supplemental data can be driven byconditional or supplemental data operations or operations that aretriggered by the results of a primary data operation, and associatedanswer phrases and sentences. For instance, a primary data operationthat yields data directly responsive to a first query intent may beassociated with the first intent and the data from that operation may beused to formulate a directly responsive answer phrase that is directlyresponsive to an oncologist’s query that pairs with the first intent. Inaddition, a second or supplemental data operation may also be associatedwith the first intent and may yield data results used to formulate asupplemental answer phrase of some type (e.g., a qualifying statement,additional data of interest in addition to the data that is directlyassociated with the initial query, clinical trials of interest,conclusions and supporting data, etc.) which, while not directlyresponsive to the first query, adds additional information of interestto the directly responsive answer phrase. Here, when the primary dataoperation yields results those results may be used to generate thedirectly responsive phrase that is responsive to the query. Similarly,when the supplemental data operation associated with the first intentyields results, those results may be used to generate a second orsupplemental response phrase. In this case, the directly responsive andsupplemental phrases may be broadcast sequentially to the oncologist tohear.

In the above case, if only the primary data operation yields a resultand associated directly responsive answer phrase (e.g., the supplementaldata operation fails to yield any data that can be used to generate asupplemental response phrase), the system would only generate thedirectly responsive phrase. Thus, in these cases, the system response toa query may include either a directly responsive phrase alone or asequence including the directly responsive phrase followed by thesupplemental phrase.

In some cases three, four, five or more supplemental data operations andanswer phrases may be associated with a single intent in the system.Here, once the intent is identified, every one of the data operations(e.g., primary and each supplemental) may be performed in an attempt toyield results that can be used to generate and broadcast a fulsomesystem response. Where only a subset of the supplemental data operationsgenerate results, only phrases associated with those results would begenerated and sequentially broadcast. Thus, for instance, in a casewhere a primary and first through fifth supplemental data operations areassociated with an intent, if the data operations yield results for theprimary, second and fifth supplemental operations, the answer wouldinclude three sequential answer phrases, a first for the primaryoperation results and second and third for the second and fifthsupplemental operation results.

A supplemental qualifying statement may be based on an inability toeffectively provide a complete answer to a query. For instance, where aprimary data operation returns fifty different effective medications fora specific cancer state, instead of broadcasting all 50 medicationsaudibly, the system may simply identify the 3 most effective medicationsand broadcast those as options along with a qualifying statement that“There are 47 other effective medications, you can say E-mail the fulllist of medications to have the full list sent to you now.”

Another type of supplemental qualifying statement may be generated by asupplemental data operation that assesses the weight of evidence thatsupports primary data operation results. For instance, where only twoprior patients with a specific cancer state responded positively to aYYY treatment, while a directly responsive query answer may indicate“There is evidence that at least some patients with the cancer staterespond positively to YYY treatment”, a supplemental response may be“Note however that only 2 patients responded positively to YYYtreatment.” In this case, the supplemental data operation would identifythe number of positively responding patients, compare that to somestatistically significant number associated with a higher level ofconfidence and, when the number is less than the statisticallysignificant number, the operation would generate the supplementalresponse as a qualifying statement. As another instance, where a primarydata operation response is “Chemotherapy is recommended for pancreaticcancer in the adjuvant setting”, a qualifying supplemental phrase maybe“However, the role of radiation is still under review in clinicalstudies.” This supplemental phrase would be generated based on resultsfrom a supplemental data operation associated with the query intent.

Other types of qualifying statements are contemplated.

Additional data of interest can be any data, subset of data, compilationof data or derivative of system data. For instance, where an oncologistasks for status of a specific patient symptom, the additional data mayinclude statuses of additional typical symptoms given a specificpatient’s current cancer state.

Supplemental responses may include detailed information related toclinical trials identified in response to a primary data operation. Forinstance, here, a directly responsive phrase to a query may be “Thereare two clinical trials that may be of interest to Dwayne Holder.” and asupplemental response may be “The first clinical trial is 23 miles fromyour office and the second trial is 35 miles from your office.” Manyother supplemental data operations regarding clinical trials arecontemplated.

In at least some cases at least some databases will include specializedclinical reports or other report types that are developed for specificpurposes where data is gleaned from EMRs and other system databases andused to instantiate specific instances of the reports for specificpatients and cancer states. Here, in at least some cases an instantiatedreport will be generated and stored in persistent form (e.g., dated andunchanging) and in other cases an instantiated report will be stored butdynamic so that the system will routinely update the report as apatient’s cancer state progresses over time. Where a report is stored inpersistent form, multiple instances of the report may be storedpersistently so that a historical record of the report can be developedover time. Where a report is stored dynamically, historical values forreport fields may be stored so that time based instances of the reportcan be subsequently generated that reflect report information at anypoint during the course of a patient’s treatment.

One advantage to using a fully formatted clinical report of a specifictype (e.g., for pancreatic cancer, for breast cancer, for melanoma,etc.) is that an oncologist that routinely uses instantiated instancesof specific report types quickly becomes familiar with types ofinformation available in the reports as well as wherein the reports theinformation resides. Once report familiarity matures, if specificinformation related to a specific patient’s cancer state is sought, theoncologist will know if that information is located in the patient’sclinical report and, once the report is accessed, where to locate thespecific information.

Another advantage associated with a clinical report is that the reportoperates as a summary of EMR data and can include additional results ofcomplex data operations on EMR data so that an oncologist does not haveto recreate or process those operations manually. Thus, the report caninclude clinically important EMR data and also data and otherinformation derived from the raw EMR data. The collaboration device 20may provide information to the oncologist that is not available on theclinical report such as TEMPUS Insights, actionable mutations, etc.

Referring now to FIGS. 17A through 17C, three pages of an exemplaryclinical report related to patient Dwayne Holder who is afflicted withpancreatic cancer are shown. The report includes all important clinicalinformation related to the patient’s cancer state including reportsections clearly marked as genomic variants, immunotherapy markers,FDA-approved therapies and current diagnosis, FDA approved therapies andother indications, current clinical trials, variants of unknownsignificance, low coverage regions, somatic variant details - clinicallyactionable, germline variant details, clinical history and oncologistnotes (see lower left field in FIG. 17A). Here, the report format issimple and clearly defined so that an oncologist can locate specificinformation of interest rapidly.

From the perspective of the present disclosure, use of formattedclinical reports as primary data sources to drive a voice basedcollaboration system eases the tasks associated with developing afulsome set of intents and supporting information for those records. Inthis regard, see again FIGS. 17A through 17C. While a large amount ofclinically important patient information is presented on the report, theamount of information is limited so that an oncologist can rapidlybecome familiar with the report format and available data. Knowing apatient’s general cancer state (e.g., pancreatic, breast, etc.) as wellas report format and report data types for that state, an oncologistwill naturally tend to limit system queries to ones calculated to beanswerable via the report type information. Because the report data islimited (albeit including all clinically important data) to a specificset of medical record data for the patient, the number of intentsrequired to support anticipated queries is appreciably limited. Forinstance, the number of intents required to fully support anticipatedqueries for the FIGS. 17A-17C report may be on the order of severalthousand as opposed to 100,000 or more for a complete EMR.

Another advantage associated with using formatted clinical reports asprimary data sources to drive a voice based collaboration system is thatthe limited number of intents required to fully support anticipatedqueries makes it much easier for the collaborative system to uniquelydistinguish an intended intent from all other supported intents. Thus,for instance, where only 5000 intents are required to fully handle allanticipated queries about information in a pancreatic clinical record,correct intent discernment is more likely than in a case where 100,000intents need to be supported.

Yet one other advantage associated with using formatted clinical reportsas primary data sources to drive a voice based collaboration system isthat the system can leverage off complex data calculations that arealready supported by an overall EMR system that generates the importantinformation in the clinical reports. Thus, in the context of pancreaticcancer, the exemplary report in FIGS. 17A through 17C already includesall clinically important data including results of complex dataoperations so that the collaboration system does not have toindependently derive required data and other information.

In some cases, near the beginning of a collaboration session, once thecollaboration system identifies a specific patient, the system willidentify the patient’s cancer state and state-specific clinical medicalrecord and automatically load up the subset of intents (e.g., “staterelated intents”) that are associated with the patient’s cancer statefor consideration. In some cases, the state related intents may be theonly intents that are considered by the system unless the oncologistinstructs otherwise. In other cases the state related intents may bepreferred (e.g., considered first or more heavily weighted options) thanother more general EMR related intents so that if first and secondintents in the state related intents and more general pool of intentsare identified as possible intended intents, the system wouldautomatically select the state related intent over the more generalintent.

In at least some embodiments data operations associated with staterelated intents will be limited to an associated clinical record. Thus,for instance, referring again to FIGS. 17A through 17C, once DwayneHolder is identified as a pancreatic cancer patient and a query intenthas been identified, in these cases the data operations would be limitedto the data and information presented in the FIGS. 17A through 17Crecord.

In other cases data operations associated with state related intents mayinclude any operations related to any EMR or other database data that isaccessible by a system processor in addition to operations directly onthe clinical report 17A-17C data and information.

In still other cases, cancer state-specific intents may be treated aspreferred intents and other more general dataset intents may only beconsidered if the system cannot identify a state-specific intent tomatch with a received query. Here, in at least some cases even when astate-specific intent is identified, the system may generate aconfidence factor associated with the intent and, if the confidencefactor is below some threshold level, may consider other more generalsystem intents as candidates to match with a specific query.

Referring now to FIG. 18 , a process 400 similar to the processdescribed above with respect to FIG. 5 is illustrated, albeit where thecollaboration system automatically limits intents to a specific cancerstate when a specific state clinical report is available for a specificpatient. While process 400 is similar to the FIG. 5 process, several ofthe FIG. 5 process steps have been eliminated from process 400 in theinterest of simplifying this explanation. For instance, FIG. 18 does notinclude steps to provide a visual response to an oncological query,among other things. Nevertheless, it should be appreciated that any ofthe additional steps shown in FIG. 5 could be added to the FIG. 18process 400 in at least some embodiments of the present disclosure.

Referring to FIG. 18 , at an initial process step 402 an EMR or othersystem stores and maintains clinical reports for specific patients andspecific cancer states (e.g., pancreatic, breast, etc.). At block 404 anadministrator uses an exemplary cancer state specific clinical reportfor each cancer state to train an essentially complete state specificset of intents and other supporting information (e.g., parameters, dataoperations and response files or phrases).

After system training, at block 406 the system monitors for activationof a collaboration device. At decision block 408, once a collaborationdevice is activated, the system monitors for voice signals and collectsany voice signal query enunciated by an oncologist. At process block 412any received utterances are transcribed to text and stored in a textfile.

Referring still to FIG. 18 , at decision block 414, a system processormonitors utterances for any information identifying a specific patient.If the oncologist does not identify a specific patient, system controlmay pass on to a process more akin to the process shown in FIG. 5 in anattempt to identify more general query intents based on a largerdataset. At block 414, if a patient is identified by an oncologist,control passes to process block 416 where the patient’s cancer state isidentified in a system database. At block 418, the system determines ifthere is a state-specific clinical record stored in a system databasefor the user. If there is no state-specific clinical record for thepatient, again, control may pass on to the process shown in FIG. 5 in anattempt to identify more general query intents based on a largerdataset.

In FIG. 18 , if a state-specific clinical record does exist for thepatient, control passes to block 420 where the system limits the pool ofintents to match with queries to the state related intents (e.g.,intents specifically associated with the patient’s state-specificclinical record type). Here, again, in some cases limitation will onlymean that some weighting factor is applied to intents which makes itmore likely the system will select a state-specific intent instead of amore general system intent. In other cases limitation means the systemwill only consider general intents until the oncologist performs someactivity which causes the system to identify state-specific intents.

In particularly advantageous cases once a patient’s general cancer state(e.g., pancreatic, breast, etc.) is determined, the system strictlylimits (e.g., considers no other intents during a query path or acollaboration session) the intent pool to match with queries to thestate specific clinical report set.

Continuing, at block 422, a processor compares a received query to thelimited intent set to identify an intent and then extracts intentrelated parameters from the query. At process block 424 the system usesthe intent and extracted parameters to define one or more dataoperations (e.g., primary or primary and supplemental per abovediscussion) to be performed on the clinical report data and, in at leastsome cases, on other accessible data sets. At block 426 the dataoperations are performed to generate information usable to respond tothe query. At block 428 response files associated with the intent anddata operations are used to formulate audio response files and at block430 the audio response files are transmitted to the collaboration deviceand broadcast to the oncologist.

In at least some cases it is contemplated that the system will supportan e-mail functionality whereby an oncologist can request e-mail copiesof different clinical record datasets or other system datasets during acollaboration session. For instance, after the system broadcastsinformation related to clinical trials that may be off interest for aspecific patient, an oncologist may enunciate “Send me informationrelated to the trials.” Here, the system would recognize theoncologist’s intent to obtain e-mails including trial information forthe trials in question, perform a data operation to access the trialinformation and then transmit that information to the oncologist’se-mail address. In addition, once the trial information is transmittedvia e-mail, the system may generate and broadcast a response to theoncologist indicating that the trial information has been sent viae-mail. In other cases it is contemplated that data and information maybe sent to an oncologist via other communication systems (e.g., as atext link, via regular mail hard copy, etc. A more complex e-mailrelated dialog path may include the following queries, where “TherapyCompany” stands in for the name of one or more companies that providetherapies, and “Therapy” stands in for the name of one or moretherapies:

-   Results of sequencing for Dwayne Holder.-   Does my patient have high TMB?-   Are they a good candidate for immunotherapy?-   What immunotherapy drugs are currently approved?-   Who manufactures Therapy?-   What are the main adverse events to Therapy?-   Email me the Therapy drug label.-   Who manufactures Therapy.-   What is the patient financial assistance phone number for Therapy    Company?-   E-mail me the Therapy Company compassionate use consent form.-   E-mail me a TEMPUS insurance reimbursement letter that my patient    Dwayne Holder has data justifying their off label use of Therapy.

In this example, the oncologist enunciates several e-mail requests whereeach would result in delivery of a different set of information to theoncologist’s e-mail account.

In at least some cases when the system receives a query via acollaboration device, data operations will be executed on data from twoor more different types of datasets. The first type may include aspecific patient’s genomic dataset that comprises details on thespecific patient’s molecular report. The second data type will includedata that resides in general knowledge database (KDB) that includesnon-patient specific information about specific topics (e.g., efficacyof specific drugs in treating specific cancer states, clinical trialsinformation, drug class - mutation interactions, genes, etc.) based onaccepted industry standards or empirical information derived by theservice provider as well as information about the service provider’ssystem capabilities (e.g., information about specific tests andactivities performed by the provider, test requirements, etc.) To thisend, see the exemplary system database 500 shown in FIG. 20 thatincludes molecular report genomic datasets and clinical data sets 502and a non-patient specific knowledge database (KDB) 504. By arrangingdata operations in this fashion, the universe of possible intents anddata operations that can be associated with any query is proscribed asdescribed above and the advantages associated with such arrangementsresult.

Referring still to FIG. 20 , datasets 502 include, among other data,genome, transcriptome, epigenome, microbiome, clinical, storedalterations proteome, -omics, organoids, imaging and cohort andpropensity data sets which are described in other patent applications insome detail. The KDB includes separate sub-databases related to specificinformation types including, as shown, provider panels 506 (e.g.,information related to genetic panels supported by the service providerthat operates the system), drug classes (e.g., drug class specificinformation (e.g., do drugs of a specific class work on pancreaticcancer, what drugs are considered to be included in a specific drugclass, etc.)), specific genes 508, immuno results (e.g., informationrelated to treatments based on specific immuno biomarker results),specific drugs, drug class-mutation interactions, mutation-druginteractions, provider methods (e.g., questions about processesperformed by the service provider), clinical trials, immuno general,clinical conditions such as clinical diseases, term sheets (e.g.,definitions of industry specific terms), provider coverage (e.g.,information about provider tests and results), provider samples (e.g.,information about types of samples that can be processed by theprovider), knowledge (e.g., scripted questions and answers on variousfrequently asked questions that do not fall into other sub-databases),radiation (e.g., information related to suitable radiation treatmentsgiven specific cancer states), NCCN guidelines (e.g., nationalguidelines related to classification of cancer states, acceptedtreatments, etc.) and clinical trials questions - answers (e.g.,information related to locations and administrators of clinical trials.Organizing the KDB into sub-databases makes it easier to manage thosedatabases as information therein evolves over time and also enablesaddition of new sub-databases related to other defined informationtypes.

To identify a genomic dataset associate with a specific patient’smolecular report, the system identifies data operations associated witha query and then associates at least one of those operations with thepatient’s genomic dataset represented on the molecular report prior toexecuting the at least one data operation on the set.

In at least some cases results of a data operation on a patient’smolecular report data inform other data operations to perform on the KDBor results from operations on a KDB inform other operations to performon a patient’s molecular report data. For instance, in a case where anoncologist queries “What are the treatment implications of DwayneHolder’s CDKN2A mutation?”, the system may associate the query with anintent. The intent may be associated with two data operations includinga first to search a general KDB for appropriate treatments for a CDKN2Amutation and a second operation to determine if the patient has alreadybeen treated with one or more of the appropriate treatments. In thiscase, results from a KDB data operation inform the molecular report dataoperation. As another instance, in a case where an oncologist queries“Did Dwayne Holder have loss of heterozygosity with his BRCA2mutation?”, the system may again identify two data operations, this timeincluding a first operation on the genomic dataset associated withDwayne Holder’s molecular report to return the patient’s loss ofheterozygosity (LOH) value and a second operation to perform on a KDB todetermine if the patient’s mutation and LOH value pairing is known to bea tumor driver. In this case, results from the operation on themolecular report data inform the KDB data operation.

Hereafter first and second exemplary processes related to handling ofthe queries “What are the treatment implications of Dwayne Holder’sCDKN2A mutation?” and “Did Dwayne Holder have loss of heterozygositywith his BRCA2 mutation?”, respectively, are described. In the interestof simplifying this explanation, the first and second processes will bereferred to as first and second examples, respectively, unless indicatedotherwise.

Referring now to FIG. 19 , a process 450 that is consistent with atleast some aspects of the present disclosure is shown that associatesdata operations with a genomic dataset represented on a patient’smolecular report prior to performing those operations on the dataset. Atprocess block 452, a collaboration device 20 (see again FIG. 1 )receives an audible query from an oncologist via the device microphonethat is related to information that appears on the specific patient’smolecular report, which can be stored in a system database. In someembodiments, the process 450 can store the specific patient’s molecularreport and/or other patient’s molecular reports in the system database.In this way, the process 450 can store molecular reports for multiplepatients. In some embodiments, the process 450 may identify the specificpatient as described in conjunction with FIG. 18 . In at least somecases, the audible query can include a question about a nucleotideprofile associated with the patient. The nucleotide profile associatedwith the patient can be a profile of the patient’s cancer. Thenucleotide profile associated with the patient can be a profile of thepatient’s germline. The nucleotide profile associated with the patientcan be a DNA profile. The nucleotide profile associated with the patientcan be an RNA expression profile. The nucleotide profile associated withthe patient can be a mutation biomarker. The nucleotide profileassociated with the patient can be a BRCA biomarker. In at least somecases, the audible query can include a question about a therapy. In atleast some cases, the audible query can include a question about a gene.In at least some cases, the audible query can include a question about aclinical data. The clinical data may include at least one of theclinical data elements described above. In at least some cases, theaudible query can include a question about a next-generation sequencingpanel. In at least some cases, the audible query can include a questionabout a biomarker. In at least some cases, the audible query can includea question about an immune biomarker. In at least some cases, theaudible query can include a question about an antibody-based test. In atleast some cases, the antibody-based test can be a a blood sample basedantibody test. In at least some cases, the audible query can include aquestion about a clinical trial. In at least some cases, the audiblequery can include a question about an organoid assay. In at least somecases, the audible query can include a question about a pathology image.The pathology image can be a slide image, for example, an imagegenerated using whole-slide imaging (WSI). In at least some cases, theaudible query can include a question about a disease type.

In some embodiments, at block 452, the process 450 can identify at leastone qualifying parameter in the audible query. In some cases, the atleast one qualifying parameter can include a patient identity, apatient’s disease state, a genetic mutation, and/or a procedure type. Insome embodiments, the process 450 can identify qualifying parameters inthe first patient’s molecular report.

At block 454 the system identifies at least one intent associated withthe audible query. Here, block 454 entails identifying a general intentas well as context parameters within the query so that a specific intentcan be formulated. For instance, in the case of the first example query“What are the treatment implications of Dwayne Holder’s CDKN2Amutation?”, a general intent identified may be “What are treatmentimplications based on gene mutation for patient?” and specific queryparameters may include “CDKN2A and “Dwayne Holder” where the underlinedgene and patient fields in the general query are populated with “CDKN2A”and “Dwayne Holder” to generate a specific query intent.

In the case of the second example query “Did Dwayne Holder have loss ofheterozygosity with his BRCA2 mutation?”, a general intent identifiedmay be “Did patient experience genetic characteristic with genemutation?” where the underlined patient, genetic mutation and genefields in the general query are populated with “Dwayne Holder”,“heterozygosity” and “BRCA2”, respectively, to generate a specific queryintent.

In at least some cases, the at least one intent can be associated withan audible query. In at least some cases, the at least one intent can bean intent related to a clinical trial. In at least some cases, the atleast one intent can be related to a drug. In at least some cases, theintent can be referred to as a drug intent if the intent is related to adrug. In at least some cases, the drug intent can be related to a drugsuch as chemotherapy. In at least some cases, the drug intent can be anintent related to a PARP inhibitor intent. In at least some cases, theat least one intent can be related to a gene. In at least some cases,the at least one intent can be related to immunology. In at least somecases, the at least one intent can be related to a knowledge database.In at least some cases, the at least one intent can be related totesting methods. In at least some cases, the at least one intent can berelated to a gene panel. In at least some cases, the at least one intentcan be related to a report. In at least some cases, the at least oneintent can be related to an organoid process. In at least some cases,the at least one intent can be related to imaging. In at least somecases, the at least one intent can be related to a pathogen. In someaspects, the pathogen may be a pathogenic mutation. In at least somecases, the at least one intent can be related to a vaccine.

The at least one intent can be related to at least one activity. The atleast one activity can include periodically capturing health informationfrom electronic health records included in the knowledge database. Theat least one activity can include checking the status of an existingclinical or lab order. The at least one activity can include ordering anew clinical or lab order. The at least one activity can includeautomatically initiating the at least one activity without anyinitiating input from the oncologist. The at least one activity caninclude uploading the patient’s EHR to the knowledge database.

Referring still to FIG. 19 , once a specific intent is identified, atblock 456 the system identifies at least one data operation associatedwith the specific intent. Here, a database correlates data operationswith intents. For instance, in some cases one or more data operationsmay be correlated with each specific intent. In other cases at leastsome data operations may depend on results from other data operations(e.g., a second operation is only performed if results from a firstoperation are within a specific value range).

In some embodiments, at block 456, the process 450 can identify the atleast one data operation based on both the identified intent and the atleast one qualifying parameter.

In the case of the first example, for the specific intent “What aretreatment implications based on CDKN2A mutation for Dwayne Holder?”,exemplary data operations may include (1) For CDKN2A mutation, searchfor appropriate treatments in a treatments KDB and (2) For appropriatetreatments, search a treatment history portion of a patient’s molecularreport genomic dataset to identify if patient already treated withappropriate treatments. Similarly, in the case of the second example,for the specific intent “Did Dwayne Holder experience loss ofheterozygosity with BRCA2 mutation?”, exemplary data operations mayinclude (1) search for LOH value in patient’s molecular report genomicdataset as well as whether the mutation is germline or somatic and (2)based on the LOH value, optionally search a KDB (e.g., the KDB 504) todetermine whether the LOH value and mutation are known to be a tumordriver.

In at least some cases, the at least one data operation can include anoperation to identify at least one treatment option. In at least somecases, the at least one data operation can include an operation toidentify knowledge about a therapy. In at least some cases, the at leastone data operation can include an operation to identify knowledgerelated to at least one drug. For example, the knowledge can be whatdrugs, if any are associated with high CD40 expression. In at least somecases, the at least one data operation can include an operation toidentify knowledge related to mutation testing. For example, theknowledge can be whether Dwayne Holder’s sample tested for a KMT2Dmutation. In at least some cases, the at least one data operation caninclude an operation to identify knowledge related to mutation presence.For example, the knowledge can be whether Dwayne Holder has a KMT2Cmutation. In at least some cases, the at least one data operation caninclude an operation to identify knowledge related to tumorcharacterization. For example, the knowledge can be if Dwayne Holder’stumor be a BRCA2 driven tumor. In at least some cases, the at least onedata operation can include an operation to identify knowledge related totesting requirements. For example, the knowledge can be what tumorpercentage TEMPUS requires for TMB results. In at least some cases, theat least one data operation can include an operation to query fordefinition information. For example, the definition information can bethe definition of PDL1 expression. In at least some cases, the at leastone data operation can include an operation to query for expertinformation. For example, the expert information can include theclinical relevance of PDL1 expression or what common risks areassociated with the Whipple procedure. In at least some cases, the atleast one data operation can include an operation to identifyinformation related to recommended therapy. For example, the informationcan be whether or not Dwayne Holder is a candidate for immunotherapygiven that he is in the 88th percentile of PDL1 expression. In at leastsome cases, the at least one data operation can include an operation toquery for information relating to a patient. In at least some cases, theat least one data operation can include an operation to query forinformation relating to patients with one or more clinicalcharacteristics similar to the patient. For example, the information canbe what the most common adverse events are for patients similar toDwayne Holder. In at least some cases, the at least one data operationcan include an operation to query for information relating to patientcohorts. For example, the information can be what the most commonadverse events are for pancreatic cancer patients. In at least somecases, the at least one data operation can include an operation to queryfor information relating to clinical trials. For example, theinformation can be which clinical trials Dwayne Holder is the best matchfor. In at least some cases, the at least one data operation can includean operation to query about a characteristic relating to a genomicmutation. In at least some cases, the characteristic can be loss ofheterozygosity. In at least some cases, the characteristic can reflectthe source of the mutation. In at least some cases, the source can begermline. In at least some cases, the source can be somatic. In at leastsome cases, the characteristic can include whether the mutation is atumor driver.

Referring again to FIG. 19 , at block 458 the system associates each ofthe at least one data operations with a first dataset (i.e., a first setof data) presented on a specific patient’s molecular report. In the caseof the first example, the system associates each of the data operationswith CDKN2A which, as seen in FIG. 17A, is presented on the molecularreport. In the case of the second example, the system associates thefirst data operation with BRCA2 and Dwayne Holder in the molecularreport genomic dataset. In some embodiments, the system can access thespecific patient’s molecular report at block 458.

In at least some cases, the first set of data can also be a gene editingtherapy that has been previously researched and/or documented by areputable source. In at least some cases, the gene editing therapy canbe a clustered regularly interspaced short palindromic repeats (CRISPR)therapy. In at least some cases, the first set of data can include apatient name. In at least some cases, the first set of data can includea patient age. In at least some cases, the first set of data can includea next-generation sequencing panel. In at least some cases, the firstset of data can include a genomic variant. In at least some cases, thefirst set of data can include a somatic genomic variant. In at leastsome cases, the first set of data can include a germline genomicvariant. In at least some cases, the first set of data can include aclinically actionable genomic variant. In at least some cases, the firstset of data can include a loss of function variant. In at least somecases, the first set of data can include a gain of function variant. Inat least some cases, the first set of data can include an immunologymarker. In at least some cases, the first set of data can include atumor mutational burden. In at least some cases, the first set of datacan include a microsatellite instability status. In at least some cases,the first set of data can include a diagnosis. In at least some cases,the first set of data can include a therapy. In at least some cases, thefirst set of data can include a therapy approved by the U.S. Food andDrug Administration. In at least some cases, the first set of data caninclude a drug therapy. In at least some cases, the first set of datacan include a radiation therapy. In at least some cases, the first setof data can include a chemotherapy. In at least some cases, the firstset of data can include a cancer vaccine therapy. In at least somecases, the first set of data can include an oncolytic virus therapy. Inat least some cases, the first set of data can include an immunotherapy.In at least some cases, the first set of data can include apembrolizumab therapy. In at least some cases, the first set of data caninclude a CAR-T therapy. In at least some cases, the first set of datacan include a proton therapy. In at least some cases, the first set ofdata can include an ultrasound therapy. In at least some cases, thefirst set of data can include a surgery. In at least some cases, thefirst set of data can include a hormone therapy. In at least some cases,the first set of data can include an off-label therapy. In some aspects,the off-label therapy can include a drug therapy. In at least somecases, the first set of data can include an on-label therapy. In atleast some cases, the first set of data can include a bone marrowtransplant event. In at least some cases, the first set of data caninclude a cryoablation event. In at least some cases, the first set ofdata can include a radiofrequency ablation. In at least some cases, thefirst set of data can include a monoclonal antibody therapy. In at leastsome cases, the first set of data can include an angiogenesis inhibitor.In at least some cases, the first set of data can include a PARPinhibitor. In at least some cases, the first set of data can include atargeted therapy. In some aspects, the targeted therapy may be amolecularly targeted therapy. In at least some cases, the first set ofdata can include an indication of use. In some aspects, the indicationof use may be an indication of use for a drug in treating a condition,such as a disease. In at least some cases, the first set of data caninclude a clinical trial. In at least some cases, the first set of datacan include a distance to a location conducting a clinical trial. In atleast some cases, the first set of data can include a variant of unknownsignificance. In some aspects, variants may be classified as pathogenic,likely pathogenic, variant of unknown significance, likely benign, orbenign variants. In at least some cases, the first set of data caninclude a mutation effect. In some aspects, the mutation effect may bepositive (e.g., associated with a reduction in risk of heart disease),negative (e.g., associated with an increase in risk of heart disease),or neutral (e.g., associated with no significant change in risk of heartdisease). In at least some cases, the first set of data can include avariant allele fraction. In some aspects, the variant allele fractionmay be the proportion of variant reads for a given mutation. In at leastsome cases, the first set of data can include a low coverage region. Inat least some cases, the first set of data can include a clinicalhistory. In at least some cases, the first set of data can include abiopsy result. In some aspects, the biopsy result may include a grade ofhow aggressive a cancer is. For example, the grade may range from one tofour, with one indicating a least aggressive cancer, and four indicatinga most aggressive cancer. In at least some cases, the first set of datacan include an imaging result. In at least some cases, the first set ofdata can include an MRI result. In at least some cases, the first set ofdata can include a CT result. In at least some cases, the first set ofdata can include a therapy prescription. In at least some cases, thefirst set of data can include a therapy administration. In at least somecases, the first set of data can include a cancer subtype diagnosis. Inat least some cases, the first set of data can include a cancer subtypediagnosis by RNA class. In at least some cases, the first set of datacan include a result of a therapy applied to an organoid grown from thepatient’s cells. In at least some cases, the first set of data caninclude a tumor quality measure. In at least some cases, the first setof data can include a tumor quality measure selected from at least oneof the set of PD-L1, MMR, tumor infiltrating lymphocyte count, and tumorploidy. In at least some cases, the first set of data can include atumor quality measure derived from an image analysis of a pathologyslide of the patient’s tumor. In at least some cases, the first set ofdata can include a signaling pathway associated with a tumor of thepatient. In at least some cases, the signaling pathway can be a HERpathway. In at least some cases, the signaling pathway can be a MAPKpathway. In at least some cases, the signaling pathway can be aMDM2-TP53 pathway. In at least some cases, the signaling pathway can bea PI3K pathway. In at least some cases, the signaling pathway can be amTOR pathway.

In at least some cases, the at least one data operations can include anoperation to query for a treatment option, the first set of data caninclude a genomic variant, and the associating step (i.e., block 458)can include adjusting the operation to query for the treatment optionbased on the genomic variant. In at least some cases, the at least onedata operations can include an operation to query for a clinical historydata, the first set of data can include a therapy, and the associatingstep (i.e., block 458) can include adjusting the operation to query forthe clinical history data element based on the therapy. In at least somecases, the clinical history data can be medication prescriptions, thetherapy can be pembrolizumab, and the associating step can includeadjusting the operation to query for the prescription of pembrolizumab.

Continuing, at block 460 the system executes each of the data operationson a second set of data to generate response data. In the case of thefirst example, the first data operation on a KDB (e.g., a second dataset) yields Palbociclib as an appropriate treatment for the patient’sCDKN2A mutation and the second data operation on the molecular reportgenomic dataset (e.g., another second dataset) indicates that DwayneHolder has already been treated with Palbociclib. In the case of thesecond example, response data from the first data operation on DwayneHolder’s molecular report genomic dataset (e.g., a second dataset)indicates no pathogenic somatic BRCA2 mutation but also indicates thatthere is a pathogenic germline BRCA2 mutation and an LOH loss associatedtherewith (see BRCA2 section of the molecular report shown at bottom ofFIG. 17B that indicates LOH). In the second example, the first dataoperation results (e.g., germline BRCA2 mutation and presence of somaticLOH) are used to drive the second data operation and the response dataindicates that the tumor is a BRCA2 driven tumor.

In at least some cases, the second set of data can include clinicalhealth information. In at least some cases, the second set of data caninclude genomic variant information. In at least some cases, the secondset of data can include DNA sequencing information. In at least somecases, the second set of data can include RNA information. In at leastsome cases, the second set of data can include DNA sequencinginformation from short-read sequencing. In at least some cases, thesecond set of data can include DNA sequencing information from long-readsequencing. In at least some cases, the second set of data can includeRNA transcriptome information. In at least some cases, the second set ofdata can include RNA full-transcriptome information. In at least somecases, the second set of data can be stored in a single data repository.In at least some cases, the second set of data can be stored in aplurality of data repositories. In at least some cases, the second setof data can include clinical health information and genomic variantinformation. In at least some cases, the second set of data can includeimmunology marker information. In at least some cases, the second set ofdata can include microsatellite instability immunology markerinformation. In at least some cases, the second set of data can includetumor mutational burden immunology marker information. In at least somecases, the second set of data can include clinical health informationincluding one or more of demographic information, diagnosticinformation, assessment results, laboratory results, prescribed oradministered therapies, and outcomes information. In at least somecases, the second set of data can include demographic informationcomprising one or more of patient age, patient date of birth, gender,race, ethnicity, institution of care, comorbidities, and smokinghistory. In at least some cases, the second set of data can includediagnosis information including one or more of tissue of origin, date ofinitial diagnosis, histology, histology grade, metastatic diagnosis,date of metastatic diagnosis, site or sites of metastasis, and staginginformation. In at least some cases, the second set of data can includestaging information including one or more of TNM, ISS, DSS, FAB, RAI,and Binet. In some aspects, the staging information may be referred toas “cancer staging information.” In at least some cases, the second setof data can include assessment information including one or more ofperformance status including at least one pf ECOG status or Karnofskystatus, performance status score, and date of performance status. In atleast some cases, the second set of data can include laboratoryinformation including one or more of type of lab (e.g. CBS, CMP, PSA,CEA), lab results, lab units, date of lab service, date of molecularpathology test, assay type, assay result (e.g. positive, negative,equivocal, mutated, wild type), molecular pathology method (e.g. IHC,FISH, NGS), and molecular pathology provider. In at least some cases,the second set of data can include treatment information including oneor more of drug name, drug start date, drug end date, drug dosage, drugunits, drug number of cycles, surgical procedure type, date of surgicalprocedure, radiation site, radiation modality, radiation start date,radiation end date, radiation total dose delivered, and radiation totalfractions delivered. In at least some cases, the second set of data caninclude outcomes information including one or more of Response toTherapy (e.g. CR, PR, SD, PD), RECIST score, Date of Outcome, date ofobservation, date of progression, date of recurrence, adverse event totherapy, adverse event date of presentation, adverse event grade, dateof death, date of last follow-up, and disease status at last follow up.In at least some cases, the second set of data can include informationthat has been de-identified in accordance with a de-identificationmethod permitted by HIPAA. In at least some cases, the second set ofdata can include information that has been de-identified in accordancewith a safe harbor de-identification method permitted by HIPAA. In atleast some cases, the second set of data can include information thathas been de-identified in accordance with a statisticalde-identification method permitted by HIPAA. In at least some cases, thesecond set of data can include clinical health information of patientsdiagnosed with a cancer condition. In at least some cases, the secondset of data can include clinical health information of patientsdiagnosed with a cardiovascular condition. In at least some cases, thesecond set of data can include clinical health information of patientsdiagnosed with a diabetes condition. In at least some cases, the secondset of data can include clinical health information of patientsdiagnosed with an autoimmune condition. In at least some cases, thesecond set of data can include clinical health information of patientsdiagnosed with a lupus condition. In at least some cases, the second setof data can include clinical health information of patients diagnosedwith a psoriasis condition. In at least some cases, the second set ofdata can include clinical health information of patients diagnosed witha depression condition. In at least some cases, the second set of datacan include clinical health information of patients diagnosed with arare disease.

Referring yet again to FIG. 19 , at block 462 the system formulates asuitable audio response file and at block 464 the response file is usedto broadcast an audible response to the oncologist. In the firstexample, the system may generate the following response “Providerrecommends Palbociclib, a CDK4/6 inhibitor based on Dwayne Holder’sCDKN2A mutation. He has already received this drug from September 20th,2017 to January 6th, 2018 however, so you may want to consider targetingone of his other clinically actionable mutations.” In the secondexample, the system may generate the following response “Dwayne Holder’sresults showed a pathogenic germline BRCA2 mutation combined with asomatic loss of heterozygosity, indicating that this may be a BRCA2driven tumor.”

It has been recognized that many different query intents may takesimilar formats where the differences between specific intents aredefined by specific parameters. Similarly, many system responses todifferent queries may have similar formats where differences between thespecific responses are defined by specific parameters in the queriesand/or results generated by data operations. For these reasons, in atleast some embodiments, a specialized user interface has been developedto reduce the burden on a system administrator associated withspecifying all possible system intents, contextual query parameters,data operations and audio response files as well as to manage thatinformation as knowledge evolves over time. The interface generatessub-databases (see sub-databases in FIG. 20 ) that form the KDB shown inFIG. 20 .

See FIG. 21 that schematically illustrates an exemplary user interfacescreen shot 520 that corresponds to the provider panels sub-database 506shown in FIG. 20 . In addition to presenting a provider panels dataset,the screen shot includes a separate selectable icon for each of thesub-database types in FIG. 20 so that an administrator can access anyone of those sub-databases via a screen shot akin to the one shown inFIG. 21 . Screen shot 520 includes a spreadsheet type arrangement ofinformation cells in rows and columns used by the system to processesqueries and generate responses as well as interface tools for scrollingup and down and left and right to access additional sub-databaseinformation. Although not shown an exemplary interface would alsoinclude a keyboard, mouse device and/or other input devices forinteracting with the interface (e.g., scrolling, modifying information,adding or deleting information, etc.)

Referring still to FIG. 21 , the screen shot 520 includes query intents522 A through ZZZ arranged in a first row of cells, a separate intent ina cell at the top of each column within a first row. Intents often takethe form of a defined query that received queries can be associatedwith. Exemplary intent A shown is “Does Provider $panel come withclinical data structuring?” where the “$panel” representation is aparameter that is gleaned from a query received from an oncologist.Although only a small number of intents are shown, it should beappreciated that hundreds or more intents may be expressed and accessedvia the interface. The $panel representation is referred to as aparameter field and the system supports many parameter types withdifferent parameter fields and any intent may include two or moredifferent parameter fields.

Referring still to FIG. 21 , parameters that may fill in the $panelparameter fields in the intents are listed in cells arranged in a lefthand column 524 on screen shot 520 and include xT, xE and xF and mayinclude many other panel types. Thus, depending on a received query(e.g., does the query reference an xT panel?), the $panel field inintent A may be filled in with any of xT, xE, xF, etc., to define apanel specific intent.

Answers are provided for each intent and parameter combination in ananswer section 526 of the screen shot. In general the answer sectionincludes separate cells for each of the parameter rows and intentcolumns and separate scripted answers may be provided in each of theanswer cells for each of the intent-parameter combinations. Forinstance, for intent C and an xT panel, the answer in an associatedanswer cell 530 is “Yes, matched normal sequencing is included in the xTpanel.”

In cases where a general answer format is applicable to each parameterin column 524, an answer format may be provided where specificparameters are used to fill in parameter fields in the answer format. Tothis end, see the answer format in field 532 that requires a panelparameter in field $panel. Here, in operation, the system retrieves asuitable panel parameter from column 524 and fills in field $panel whenappropriate. Although not shown in FIG. 21 , a negative answer row 536is also provided that may include negative answer formats for one oreach of the intents listed in row 522.

Referring still to FIG. 21 , an administrator can change any intent, addintents, delete intents, change a parameter in column 524, add aparameter, delete a parameter and/or change an answer by simplyselecting an instance of the information to change and then typingdifferent information into the associated cell. In this way, intents andanswers with formats that are similar for different parameters can bequickly specified and managed with less overall effort. For instance, inFIG. 21 assume the interface specifies 200 different intents and anadministrator wants to add a new panel to the parameter options. Here,the administrator can just select another cell in the parameter columnand name the new panel causing all the intents in row 522 to beassociated with the new panel name. In addition, when the new panel isadded to the panel column, for each answer format (e.g., see again 532)that remains valid for the new panel, that answer formats areautomatically applied to the new panel.

Referring now to FIG. 22 , a second administrator interface screen shot550 is illustrated that has a format similar to the FIG. 21 providerpanels screen shot and, to that end, includes an intents row 552, ananswer section 554 and a parameters section 556. Each exemplary intentincludes a parameter field $Gene which is filled in with one of theparameters from the parameter column 556 that forms part of a receivedquery.

In FIG. 22 the answer section 554 is different than in FIG. 21 as“answer values” are provided in each answer cell (e.g., a cellcorresponding to a specific intent column and parameter row combination)that are used in at least one and in some cases two different ways.First, answers in the answer cells corresponding to specific intent andparameter pairings can be used to select one of the answer format 551 ornegative answer format 553. To this end, each of the answer format andthe negative answer format for each format includes each of a rule and aresponse format where the rules apply based on answer cell values. Thus,for instance, for the answer format in cell 560, the rule is “IF TRUE”(e.g., if a TRUE value is in an answer cell), then apply the associatedanswer format. Similarly, for the negative answer format in cell 562,the rule is “IF FALSE″(e.g., if a FALSE value is in an answer cell),then apply the associated negative answer format. Thus, for instance,because the answer cell 570 includes the value TRUE for gene ABCB1 andintent A, the answer format in cell 560 is applied and the response fileincludes the phrase “Yes, Provider sequences ABCB1.” Similarly, becausethe answer cell 572 includes the value false for gene ABCB4 and intentA, the negative answer format in cell 562 is applied and the responsefile includes the phrase “No, Provider does not sequence ABCB4.”

Second, in at least some cases answer cell values can also be used topopulate one or more fields in an answer format or a negative answerformat. To this end, see for instance the answer format in cell 576which, in addition to including a $Gene field, also includes an $AV(e.g., answer value) field. Here, when the answer format rule is met(e.g., IF AV; if there is an answer value in an answer cell) so thatanswer format 576 is used to generate a response file, in addition topopulating the $Gene field with one of the genes from column 556, the$AV field is populated with a value from an associated answer cell therebelow. For instance, for gene ABCB1 the answer cell 578 includes a value1% and therefore, if intent C applies and is qualified by gene parameterABCB1 the answer format rule in cell 576 is met and the response tileincludes the phrase “Provider sees a pathogenic mutation in ABCB1 in 1%of pancreatic cancer patients”. In negative answer cell 580, the rule isthat if an answer cell there below is blank, then that cell format isused to generate a response file.

While there are two answer format rows shown in each of FIGS. 21 and 10(e.g., the answer format row and the negative answer format row), inother cases there may be three or more answer formats that change basedon values in specific answer fields there below to support more complexanswer generation schemes.

Again, as in the case of the data presented in FIG. 21 , the data inFIG. 22 only shows a small subset of the gene data accessible via leftand right and up and down scrolling through parameters and intents. Forinstance, the genes in parameter column 556 may include an entire genepanel (e.g., hundreds of genes) and the intents in row 552 may includehundreds or even thousands of intents.

FIG. 23 shows another administrative screen shot 600 similar to the FIG.21 and FIG. 22 shots, albeit corresponding to a provider methods dataset. The spreadsheet representation in FIG. 23 is similar to therepresentations in FIGS. 21 and 22 including an intent row 602, ananswer format section 610, and a parameters column 604. One differencein FIG. 23 is that the first intent A includes two parameter fields andthe parameters section includes first and second parameter rows, one foreach of the parameter fields in intent A. More specifically, theparameters section include a first column listing tests and a secondcolumn that lists test methods for populating associated $test and$testmethod fields in the intent statement. In addition, in at leastsome cases answer formats like the negative answer format shown in cell606 will include two or more parameter or value fields. Here operationis similar to that described above, albeit using two parameters toinstantiate specific intents and final response files.

Referring again to FIG. 20 , interface screen shots akin to thosedescribed in FIGS. 21 through 23 are included in a system for specifyingintents, parameters and answer formats for each of the information typesassociated with the sub-databases illustrated. Some of the screen shotswill include specific scripted answers for specific intents while otherswill rely upon answer formats, rules for one or all the formats andpopulating answer fields with intent parameters and/or database valuesthat appear in answer cells as described above. Other screen shots andtool combinations are contemplated.

In at least some cases it is contemplated that the system will enable anoncologist to request visual access to query answers and/or relatedinformation (e.g., associated documents (e.g., clinical trialinformation, drug label warnings, etc.). For instance, an oncologist mayenunciate “Make that answer available with the system web platform,”causing the system to render the most recently broadcast answeravailable via a nearby or oncologist dedicated computer display screen.In at least some cases it is contemplated that the system will enable anoncologist or other user to provide queries via a typed question insteadof an audible query. For instance, rather than speaking a question, anoncologist may type the query into a mobile phone or other computingdevice, and the query may be processed as described herein.

Referring now to FIG. 24 , a fourth exemplary system 650 including amobile device 652 is depicted. The fourth exemplary system can includethe collaboration device 20, the collaboration server 12, the AIprovider server 14, and the database 18. The collaboration device 20,the collaboration server 12, the AI provider server 14, and the database18 can be linked together as described above in conjunction with FIG. 1. The mobile device 652 can be used in conjunction with thecollaboration device 20 to authenticate user credentials and/or onboardthe oncologist, as well as perform at least a portion of the functionsof the collaboration device 20 (e.g., handle queries about a patient)among other suitable uses.

The mobile device 652 can be a smartphone, a tablet, or another suitablemobile computing device. The mobile device 652 can include a camera 653,a speaker 654, a fingerprint sensor 656, and input button 658, and atouchscreen 660, Similar to the collaboration device 20, the mobiledevice 652 can transmit 666 voice signal messages to the transceiver 16for processing by the collaboration server 12 and/or AI provider server14. The mobile device 652 can also receive 662 visual response filesand/or receive 664 audio response files generated based on the voicemessage signals from the transceiver 16.

In addition, the mobile device 652 can transmit 668 authenticationinformation to the transceiver 16 in order to unlock the collaborationdevice 20. The collaboration device 20 may be configured to requestauthentication from the oncologist at predetermined time points (e.g.,every thirty minutes, every hour, etc.) or when the oncologist moves thecollaboration device 20. For example, the collaboration device 20 candetect it has been moved if contact with the transceiver 16 is lost. Theoncologist may have moved the collaboration device to another room inthe same building (e.g., a hospital) or to another building entirely(e.g., another hospital, a home office, etc.). It is appreciated thatthe collaboration device 20 is mobile and can be moved to and used in avariety of locations with suitable connectivity (e.g., wirelessinternet).

The mobile device 652 can have a mobile device application (not shown)installed that can determine what authentication credentials arerequired at a specific time and display notification about requiredauthentication credentials on the touchscreen 660 when applicable. Forexample, if the mobile device application determines that a half hourhas passed since the last authentication, the mobile device applicationcan output a notification that the oncologist needs to reauthenticatebefore using the full capabilities of the collaboration device 20 (e.g.,querying the collaboration device 20 about a specific patient). In someembodiments, the mobile device application may not output anotification, and the collaboration device 20 can prompt the oncologistto reauthenticate when the oncologist attempts to query thecollaboration device.

The mobile device 652 can provide 668 multiple forms of authenticationinformation to the transceiver. The authentication information caninclude a fingerprint scan generated using the fingerprint sensor 656, apicture of the face of the oncologist generated using the camera 653,and/or a text password. In some embodiments, the mobile deviceapplication can provide the raw fingerprint scan, the picture, and/orthe password to the transceiver 16, and another process (e.g., a processin the collaboration server 12) can determine if the authenticationinformation is sufficient (e.g., the fingerprint scan sufficientlymatches a predetermined fingerprint scan associated with the oncologist)or not. In other embodiments, the mobile device application candetermine if the authentication information is sufficient or not, andtransmit 668 authentication information indicating whether theauthentication information is sufficient or not (e.g., a Booleanyes/no). If the authentication information is sufficient, thecollaboration device 20 can resume full operation.

The mobile device 652 can also transmit 669 a user request (i.e., anoncologist request) to the transceiver for processing by thecollaboration server 12. The user request can be a request for a human(e.g., an administrator or in some cases, a medical practitioner) toreview a note, clinical report, molecular report, patient, case, etc., aproduct suggestion (e.g., a collaboration device 20 capability theoncologist would like to add), product fulfillment requests (e.g., anorder for testing kits), a status of an ordered test kit (e.g., a liquidor tissue based biopsy test kit for executing a molecular test), arecommendation for a tumor board session for a patient, or otherrequests that may be more easily disseminated by a human than a computerprocess or requests not necessarily related to intent fulfillment. Someuser requests may be generated by and transmitted from the mobile device652 and/or the collaboration device 20 based on a voice signal capturedfrom the oncologist. The AI database 14 can determine the intent of theuser request to be an order for testing kits or a request for manualreview of a particular case, for example. The text form of the voicesignal and any associated information (e.g., the intent) can then betransmitted to the collaboration server 12, which can then transmit thetext form of the voice signal and any associated information to anadministrator and/or a suitable computer process. If the user requestincludes a recommendation of a patient for a tumor board or a clinicaltrial, relevant information about the patient can also be transmittedalong with the request, greatly reducing the need for filling outapplications for clinical trials and/or tumor boards.

In some embodiments, the processes performed by the mobile device 652(e.g., authentication processes, transmitting 669 user requests, etc.)can be performed by the collaboration device 20.

Referring now to FIG. 24 as well as FIG. 25 , a mobile applicationscreen shot 700 is shown. The mobile application screen shot 700 can bea portion of the mobile device application included in the mobile device652. The mobile application screen shot 700 can include a battery levelindicator 702 indicating a battery level of the collaboration device 20,a username 703 corresponding to the current oncologist that is loggedin, an authentication indicator 704 indicating whether or not theoncologist has been authenticated by the mobile device 652, a microphonebutton 706, a night mode button 708, and a mute button 710. Theoncologist can select the microphone button 706 instead of enunciating awake word or phrase (e.g., “TEMPUS ONE”) in order to prompt thecollaboration device 20 and/or the mobile device 652 to record a voicesignal. The night mode button 708 can control the darkness and/or colorsdisplayed by the mobile device application.

The mobile application screen shot 700 can include a slider 712. Theoncologist can actuate the slider 712 to control a volume of thecollaboration device 20. The mobile application screen shot 700 caninclude a suggested questions section 714 that can display examplequeries and/or common queries that oncologists ask the collaborationdevice 20. For example, a first question 716 can show the oncologist howto inquire about a specific patient, and a second question 718 can showthe oncologist how to inquire about medical questions not specific tothe patient.

Referring now to FIGS. 24 and 25 as well as FIG. 26 , a second mobileapplication screen shot 720 is shown. The second mobile applicationscreen shot 720 can include the suggested questions section 714, thefirst question 716, and the second question 718 included in the mobileapplication screenshot 700 shown in FIG. 25 . In FIG. 26 , the suggestedquestions section 714 is shown to include additional suggestedquestions. The second mobile application screen shot 720 can include asuggest new capabilities button 722. The oncologist can select thesuggest new capabilities button 722 and provide a suggestion about newcapabilities in a popup box, for example. The mobile device 652 can thentransmit the suggestion to an administrator.

The second mobile application screen shot 720 can include a frequentlyasked questions (FAQs) section 724 that includes common questions aboutthe functionality of the collaboration device 20. The second mobileapplication screen shot 720, and more specifically the FAQs section 724,can include a search button 726 that the oncologist can select in orderto search a set of FAQs.

Referring now to FIG. 25 as well as FIG. 27 , a third mobile applicationscreen shot 730 is shown. The third mobile application screen shot caninclude an answer 732 to the second question 718 shown in FIG. 25 . Theanswer 732 can include text and may be included in a popup box that isdisplayed when the oncologist selects the second question 718.

Referring now to FIG. 28 , a fifth exemplary collaboration system 750 isshown. The fifth exemplary system 750 can include an administratordevice 752 such as a laptop or desktop computer. An administrator canuse the administrator device 752 to analyze data aggregated from anumber of oncologists, update firmware in the collaboration device 20,analyze requests from an oncologist, update a set of intents (e.g., inDIALOGFLOW), and other suitable tasks related to operation of thecollaboration device and/or the mobile device 652.

The fifth exemplary system 750 can include a cloud architecture 754 thatincludes a number of modules that may be located remotely (e.g., on oneor more servers) in relation to the administrator device 752,collaboration device 20, and/or the mobile device 652. The collaborationdevice 20 and/or the mobile device 652 can be linked to an IoT coremodule 758 that can handle authentication requests and othercommunications from the collaboration device 20. The IoT core module 758can be linked to a pub/sub module 760 that is linked to anauthentication module 762 and a ping module 764. The pub/sub module 760can transmit updates (e.g., status updates) to the collaboration device20 and/or the mobile device. The authentication module 762 can receiveauthentication requests from the collaboration device 20 and/or themobile device 652. The pub/sub module 760 can direct communications fromthe IoT core module 758 to either the authentication module 762 or theping module 764 as appropriate. The pub/sub module 760, the IoT coremodule 758, the authentication module 762, and/or the ping module 764can be stored on a first server 756.

The collaboration device 20 and/or the mobile device 652 can be linkedto a gateway module 768 that may be included in a second server 766. Thegateway module 768 can include at least a portion of the processesincluded in the collaboration server 12. The collaboration device 20and/or the mobile device 652 can transmit requests (e.g., a user requesttransmitted from the mobile device 652) and/or voice signal messages tothe gateway module 768. The gateway module 768 can be linked to an AImodule 774. The AI module 774 can include at least a portion of theprocesses included in the AI provider server 14 (e.g., voice signalextraction processes) and may receive voice signals, extract intentsfrom the voice signals, and transmit data response to the gateway module768 for transmitting to at least one of the collaboration device 20 andthe mobile device 652. In some embodiments, the DIALOGFLOW suite can beincluded in the AI module 774. The gateway module 768 can also be linkedto a debug bucket module 778 and a redis module 780 included in a thirdserver 776. The gateway module 768 can be linked to an AI demo module772 included in a fourth server 772.

The third server 776 may only be fully accessible (e.g., configured toallow full control and/or modification of processes) by theadministrator device 752 and not the collaboration device 20 and/or themobile device 652. The third server 776 can include an administratormodule 782 that the administrator device 752 can access in order toupdate collaborator device 20 firmware, define intents, update intentfulfillment processes, and perform other administrator functions. Theadministrator module 782 can also process and/or transmit user requests(e.g., an order for testing kits) to the administrator device 752. Theadministrator and/or the administrator device 752 can then analyze theuser request and proceed accordingly. For example, an order for testingkits can be transmitted to an order fulfillment center. As anotherexample, the oncologist can request a manual review of a particularcase, and the request can be transmitted to an administrator who mayassign the case to a medical practitioner for review within apredetermined timeframe, for example, twenty-four hours.

The third server 776 can include a console module 786 linked to theadministrator device 752. The console module 786 can performadministrative duties. An administrative database 784 can be linked tothe administrator module 782. The console module 786 can be linked to aportal module 790 included in a fifth server 788. The portal module 790may provide an interface for oncologists to review molecular testsreports.

The fifth system 750, and more specifically the administrator module782, the administrator device 752, the console module 786, and/or theadministrator database 784 may track a number of collaboration devices20. More specifically, the fifth system 750 can track if eachcollaboration device 20 is connected (e.g., in contact with the cloudarchitecture 754) or active (e.g., processing a query), what version offirmware each collaboration device 20 is running, and relatively staticinformation such as an owner and/or institution associated with thedevice.

The administrator module 782 can include processes for analyzing queriesfrom oncologists and generate usage data about how oncologists are usingthe collaboration device 20, how many test kits are being ordered fordifferent case types, how often questions about specific sections ofgenerated clinical reports are asked about FDA on/off label drugquestions, therapies associated with a specific variant, what actionsoncologists take in different scenarios (e.g., what questions are beingasked), and other suitable data.

It is understood that the servers 756, 766, 770, 776, and 788 can eachinclude more than one server. Additionally, at least some of the modulesand/or processes included in the cloud architecture 754 can beimplemented with infrastructure-as-code that can be migrated acrossclouds like AWS, GOOGLE CLOUD, AZURE, etc.

The fifth system 750, and more specifically the administrator module782, the administrator device 752, the console module 786, and/or theadministrator database 784 can track intents being processed across anumber of cases (e.g., thousands of cases) and/or other actions one ormore collaboration devices 20 are taking or developments being made inthe medical community (e.g., new research articles, studies, and/ortreatment techniques) and provide “nudges” to oncologists in order topotentially make the oncologists aware of information they potentiallymay not be aware of. The fifth system 750 may ask the oncologist forpermission to analyze clinical data generated by the oncologists.

Other data the administrator module 782, the administrator device 752,the console module 786, and/or the administrator database 784 can trackmay include numbers of test kits ordered by an oncologist in apredetermined timeframe (which may indicate if onboarding issuccessful), how many answers (e.g., statistics) or other information(TEMPUS Insights, actionable mutations, etc.) the collaboration devices20 provide that are not included in clinical reports, periodicallyscheduled surveys of oncologists about the collaboration device 20and/or the mobile application (e.g., answers to “What information wouldbe most helpful when making clinical decisions?”), what portions ofclinical reports are asked about the most often (either for anindividual oncologist or for multiple oncologists), how oncologistsbehave on a macro scale (e.g., for a given cancer type and/or molecularvariation, what tests did other oncologists run), how similar patientsbehaved to certain therapies (e.g., how many patients presented with XXXmolecular mutation had variant YYY, and of those, how many had ZZZresponse to therapy AAA over time duration BBB), or other suitable data.

In some embodiments, the administrator device 752 may provide at leastsome of the functionalities of the collaboration device 20, albeittailored for the administrator. For example, the administrator device752 may be suitable equipped (e.g., with a microphone and speakers) andconfigured to answer questions such as “where is sample [x] stored?”,“what is the SOP for scenario y”, etc. that may only be relevant to theadministrator. In some embodiments, the administrator may use thecollaboration device 20 with an administrator-specific set of intentsthat oncologists may not be able to use (i.e., are restricted fromusing). The administrator may enunciate e.g. TEMPUS ONE, where is sample[x] stored?” and the collaboration device can determine the intent ofthe query is to know the location (e.g., a warehouse) of the sample [x],and provide an appropriate visual and/or audible response.

Some data can then be used to customize the user experience of eachoncologist. For example, data about what portions of clinical reportsare the most asked about can be used to custom tailor report layout andformat suggestions that an oncologist could accept (i.e., update thereport layout and/or format) or reject (i.e., keep the same reportlayout and/or format) after receiving the notification. Then, reportsdisplayed on the portal module 790 or via the mobile application (e.g.,on the touchscreen 660) can follow the updated template and/or layout.Additionally or alternatively, the oncologist can provide suggestionsabout the report layout and/or format, and the report layout and/orformat can be updated accordingly.

The fifth system 750 can provide nudges to the oncologist using thecollaboration device 20 and/or the mobile device 652 using datacollected by the fifth system 750. The nudges can be provided to anoncologist without the oncologist needing to ask a question. One nudgecan include the fifth system 750 determining a treatment that is mostsuccessful for patients similar to a patient the oncologist isanalyzing. The most successful treatment can be determined based onmolecular data of the patient (e.g., molecular mutations and/orvariants), age, gender, etc. as well as the success rates of varioustreatments in populations with the same molecular data, age, gender,etc. Another nudge can include informing the oncologist of cancerboards, clinical trials, and other programs within a predeterminedradius (e.g., fifteen miles of the medical facility the oncologist islocated at) the patient is eligible for. Furthermore, the fifth system750 can provide the nudge to the oncologist at a predetermined timebefore an upcoming patient visit (e.g., within twenty-four hours), andmay only provide cancer boards, clinical trials, and other programs thatwere not available when the patient last visited and/or the lastclinical report was generated for the patient. The oncologist can benotified by controlling the indicator lights 50 in a predeterminedpattern and/or color, outputting specific sounds at the speakers 44,displaying notifications on the mobile device 652, vibrating thecollaboration device 20 and/or mobile device 652 using a hapticsignaling component, etc. Additionally, the findings of any tumor boardsand/or trials and/or any action plans can be provided to the oncologistor cite a specific action plan following a tumor board. The oncologistcan then easily retrieve findings of a given tumor board.

Yet another nudge can include controlling the indicator lights 50 in apredetermined pattern and/or color, outputting specific sounds at thespeakers 44, displaying notifications on the mobile device 652,vibrating the collaboration device 20 and/or mobile device 652 using ahaptic signaling component, etc., to indicate that a new molecularreport or clinical report has become available for the patient.

Still another nudge can include notifying the oncologist ofnewly-available content (e.g., research papers, articles, journals,posters, etc.) that is relevant to the practice area of the oncologistor for the patient. Oncologists can opt in for notifications associatedwith multiple data sources, content types, cancer subtypes and/ordiseases, molecular mutations/variants, treatments (FDA on-label,off-label, investigational, etc.) as well as clinical trials. A furthernudge can include notifying the oncologist that an ordered test may becompleted more efficiently with an alternative test (e.g., using an xFliquid biopsy test instead of a tissue-specific xT panel). The test kitmay not be able to be processed due to insufficient tissue, but theoncologist will only see the newly-suggested test.

A still further nudge can include notifying an oncologist of variouspatient tests and orders that other oncologist have made for similarpatients and/or cases. For example, the nudge can include a notificationthat a peer oncologist (or x% of other oncologists) has placed a testorder for a similar patient. In some embodiments, the fifth system 750can determine if oncologists have ordered PDL1 IHC with a specific test,and inform the oncologist whether or not the test is a good option forthe patient. Sometimes, oncologists do not want to be the first to use anew vendor and may also want to consult their peers to better understandthe type of information returned via PDF report, through the portalmodule 790 or the mobile application on the mobile device 652. Knowingthat other oncologist peers in an institution are ordering tests for x%of certain patient cohorts may allow the oncologist to operate with aknowledge of how other oncologists are treating similar patients.

Still regarding test kits, yet another nudge can include notifying theoncologist that their stock of test kits is running low (e.g., below apredetermined threshold). Some nudges can include information aboutfinancial assistance that may be available for a test kit. Once anoncologist is informed of test kit options, the oncologist can order thetest kit and/or apply for financial aid by enunciating an appropriatecommand to the collaboration device 20 and/or the mobile device 652. Thefifth system 750 may then automatically fill out any test kit orderforms and/or financial aid applications.

Some nudges can inform an oncologist of nearby continuing medicaleducation (CME) courses and/or allow the oncologist to enroll inCME-crediting courses or highlight local and/or online offerings withinthe particular specialty and/or area of focus of the oncologist.

Referring now to FIGS. 19, 24, and 28 as well as FIG. 29 , a process1000 for generating supplemental content for a physician based on amolecular report associated with a specific patient is shown. Theprocess 1000 can be used to provide patient-specific nudges to theoncologist. The process 1000 can identify information that may berelevant to treatment of a patient and that the oncologist may not haveconsidered when querying the collaboration device 1000. In this way, thecollaboration device 20 may assist the oncologist in treating thepatient using therapies, drugs, clinical trials, and/or other treatmenttechniques applicable to the specific patient that the oncologist maynot be aware of or may not have considered previously. The process 1000may also provide the oncologist 1000 with information about how otheroncologists have treated similar patients (e.g., similar gnomicallyand/or being diagnosed with a similar cancer type). The process 1000 maybe executed by a suitable system such as the fifth exemplary system 750.

At 1002, the process 1000 can determine a specific patient. In at leastsome cases, the process 1000 can be executed in parallel and/or afterthe process 450 is being executed and/or after the process 450 has beenexecuted. The process 1000 can determine that the specific patient isthe same specific patient identified by the process 450. In at leastsome cases, the process 1000 can be executed along with the process 1000to effectively form a single process. The process 1000 can then proceedto 1004.

At 1004, the process 1000 can store and maintain a general cancerknowledge database. The general cancer knowledge database can includeraw data and/or processed data about a number of patients includingmolecular reports, presence of conditions such as diabetes, heartdisease, etc., information about treatment history such as drugs and/ortherapies that each patient has taken as well as responses to the drugsand/or therapies (e.g., a patient was successfully treated using drugFFF), and/or other suitable data about patients. Data associated witheach patient can be persistently updated as additional informationbecomes available. The general cancer knowledge database can includenon-patient specific information about specific topics (e.g., efficacyof specific drugs in treating specific cancer states, clinical trialsinformation, drug class - mutation interactions, genes, etc.) based onaccepted industry standards or empirical information derived by theservice provider as well as information about the service provider’ssystem capabilities (e.g., information about specific tests andactivities performed by the provider, test requirements, etc.) Thegeneral cancer knowledge database can include the KDB 504 describedabove. The general cancer knowledge database can include informationabout available clinical trials, treatments, studies, academic papers,CLE courses, or other available resources. The process 1000 can thenproceed to 1006.

At 1006, the process 1000 can persistently update the specific patient’smolecular report. For example, the process 1000 can update relevantclinical trials included in the molecular report. The process 1000 canthen proceed to 1008.

At 1008, the process 1000 can automatically identify at least one intentand associated data operation related to the general cancer knowledgedatabase based on the specific patient’s molecular report data. The atleast one intent can be related to drugs, genes, testing methods, etc.as described above. The at least one intent can also be related to aspecific cancer the specific patient has been diagnosed as having. Atleast some of the intents may be intents the oncologist has not queriedthe collaboration device 20 about previously. The process 1000 can thenproceed to 1010.

At 1010, the process 1000 can persistently execute the associated dataoperation on the general cancer knowledge database to generate a new setof response data not previously generated. In some cases, the process1000 can persistently execute multiple associated data operations on thegeneral cancer knowledge database. Persistently executing the generalcancer knowledge database can allow the process 1000 to provide updatedinformation (i.e., the new set of response data) to the oncologist.Additionally, the new set of response data may provide be used toprovide information related to the specific patient that the oncologistmay not have been aware of previously. For example, the new set ofresponse data can be used to inform the oncologist of how varioustreatment options perform on other patients with similar genomicprofiles. As another example, the new set of data can be used to informthe oncologist of tests that were ordered for other patients diagnosedwith the same cancer as the specific patient and that have similargenomic profiles (e.g., the presence of a specific gene mutation). Theprocess 1000 can then proceed to 1012.

At 1012, the process 1000 can, upon generating a new set of responsedata, use the new set of response data to generate a notification tooutput to the oncologist. In some cases, the notification can be anaudible response file the process 1000 generates based on the new set ofresponse data. In some cases, the notification can be a visual indicatorthe process 1000 generates based on the new set of response data. Thevisual indicator can include a question related to the new set ofresponse data. For example, the question can be a suggested questionthat can be answered using the new set of response data. In thisexample, if the new set of response data includes information aboutpatient response to a certain treatment (e.g., patients with mutationXXX that received treatment YYY survived cancer type WWW VVV% of thetime), the suggested question can be “How many patients with mutationXXX survived when given treatment YYY?” or “What treatment is mosteffective for patients with mutation XXX and cancer type WWW?” Theprocess 1000 can then proceed to 1014.

At 1014, the process 1000 can output the notification generated at 1012to the oncologist. If the notification is an audible response file, theprocess can output the audible response file at the collaboration device20 (e.g., at the speakers 44) and/or the mobile device 652 (e.g., at thespeaker 654). If the notification is a visual indicator, the visualindicator can be output at the collaboration device 20 (e.g., at thedisplay screen(s) 48) and/or the mobile device 652 (e.g., at thetouchscreen 660). If the visual indicator is a suggested question, thesuggested question can be displayed in the suggested question section714 described above. The notification can function as a nudge. Thus, theprocess 1000 can generate and provide at least some of the nudgesdescribed above to the oncologist.

Referring now to FIGS. 19, 24, and 28 as well as FIG. 30 , a process1050 for generating non-patient-specific supplemental content for aphysician is shown. The process 1050 can be used to providenon-patient-specific nudges to the oncologist. The process 1050 canidentify information that may be generally relevant to the oncologist,such as newly available treatments, studies, academic papers, etc. Theprocess 1050 can reduce the need for the oncologist to search for newdevelopments in the field(s) the oncologist practices in. For example,if the oncologist specializes in treating breast cancer patients, theprocess 1050 may provide information that is may be useful for treatingbreast cancer patients. The process 1050 may be executed by a suitablesystem such as the fifth exemplary system 750.

At 1052, the process 1050 can determine one or more interest streams forthe oncologist. The interest streams can include newly availableclinical trials, treatments, studies, academic papers, CLE courses, orother suitable types of information and/or programs related to a cancertype that may be useful to the oncologist. In some embodiments, theoncologist can provide (e.g., audibly) the types of interest streamsand/or cancer types of interest to the process 1050. The process 1050may automatically determine the interest streams based on the history ofthe oncologist. For example, the oncologist may generally treat breastcancer and lung cancer patients, and the process 1050 can selectinterest streams available that are related to those cancer types. Theprocess 1050 can then proceed to 1054.

At 1054, the process 1050 can store and maintain a general cancerknowledge database. The general cancer knowledge database can includeraw data and/or processed data about a number of patients includingmolecular reports, presence of conditions such as diabetes, heartdisease, etc., information about treatment history such as drugs and/ortherapies that each patient has taken as well as responses to the drugsand/or therapies (e.g., a patient was successfully treated using drugFFF), and/or other suitable data about patients. Data associated witheach patient can be persistently updated as additional informationbecomes available. The general cancer knowledge database can includenon-patient specific information about specific topics (e.g., efficacyof specific drugs in treating specific cancer states, clinical trialsinformation, drug class - mutation interactions, genes, etc.) based onaccepted industry standards or empirical information derived by theservice provider as well as information about the service provider’ssystem capabilities (e.g., information about specific tests andactivities performed by the provider, test requirements, etc.) Thegeneral cancer knowledge database can include the KDB 504 describedabove. The general cancer knowledge database can include informationabout available clinical trials, treatments, studies, academic papers,CLE courses, or other available resources. The process 1050 can thenproceed to 1056.

At 1056, the process 1050 can automatically identify at least one intentand associated data operation related to the general cancer knowledgedatabase based on the interest streams associated with the oncologist.For example, the at least one intent can be related to identifyingwhether or not any new academic papers about a certain cancer type(e.g., breast cancer) are newly available (e.g., published in the lastweek), identifying whether or not any new clinical trials for a certaincancer type (e.g., lung cancer) are newly available, identifying whetheror not any new treatment options for a certain cancer type (e.g., breastcancer) are newly available, or other suitable intents. In this example,the associated data operations can include searches for any newavailable academic papers, clinical trials, and treatment options. Atleast some of the intents may be intents the oncologist has not queriedthe collaboration device 20 about previously. The process 1050 can thenproceed to 1058.

At 1058, the process 1050 can persistently execute the associated dataoperation on the general cancer knowledge database to generate a new setof response data not previously generated. In some cases, the process1050 can persistently execute multiple associated data operations on thegeneral cancer knowledge database. Persistently executing the generalcancer knowledge database can allow the process 1050 to provide updatedinformation (i.e., the new set of response data) to the oncologist. Thenew set of data can be used to inform the oncologist of newly availableacademic papers, clinical trials, and treatment options, etc. that areavailable. The process 1050 can then proceed to 1060.

At 1060, the process 1050 can, upon generating a new set of responsedata, use the new set of response data to generate a notification tooutput to the oncologist. In some cases, the notification can be anaudible response file the process 1050 generates based on the new set ofresponse data. In some cases, the notification can be a visual indicatorthe process 1050 generates based on the new set of response data. Thevisual indicator can include a question related to the new set ofresponse data. For example, the question can be a suggested questionthat can be answered using the new set of response data. In thisexample, if the new set of response data includes information about anewly available breast cancer treatment (e.g., treatment YYY is nowavailable for breast cancer patients), the suggested question can be“Are there any new treatment options available for breast cancerpatients?” The process 1050 can then proceed to 1062.

At 1062, the process 1050 can output the notification generated at 1060to the oncologist. If the notification is an audible response file, theprocess can output the audible response file at the collaboration device20 (e.g., at the speakers 44) and/or the mobile device 652 (e.g., at thespeaker 654). If the notification is a visual indicator, the visualindicator can be output at the collaboration device 20 (e.g., at thedisplay screen(s) 48) and/or the mobile device 652 (e.g., at thetouchscreen 660). If the visual indicator is a suggested question, thesuggested question can be displayed in the suggested question section714 described above.

Referring now to FIG. 31 , a process 800 that may be used for onboardingan oncologist is shown. At block 802, the process 800 can determine auser (e.g., an oncologist) has opened a mobile application, and that acollaboration device is on. The mobile application can be the mobileapplication included on the mobile device 652, and the collaborationdevice can be the collaboration device 20 described above. Thecollaboration device 20 may output “Hello, your TEMPUS ONE is ready forsetup. Please download the TEMPUS ONE mobile app to begin setup” usingthe speakers 44. Control passes to block 804, where the process 800 candisplay an option to review high level instructions to the oncologist.The option can be displayed on a user interface such as the touchscreen660. Control passes to block 806, where the process 800 can determine ifthe oncologist has logged in to the mobile application.

Once the oncologist has logged in, the control passes to block 808,where the process 800 can attempt to log the mobile device 652 in to awireless network that the collaboration device is connected to. Afterthe mobile device 652 logs in to the wireless network, the controlpasses to block 810, where the process 800 displays an option toconfigure security settings to the oncologist at the user interface.Control passes to block 812, where the process 800 proceeds to block 814if the oncologist selects the option to configure security settings(i.e., the “YES” at block 812). If the oncologist does not select theoption to configure security settings (i.e., the “NO” at block 812),control passes to block 816. At block 814, the process 800 can configurethe security setting of the mobile device 652 and/or the collaborationdevice 20. For example, the process 800 can set an authenticationpreference for the oncologist (e.g., a fingerprint preference, a faceidentification preference, or a typed password preference).

Flow then passes to block 816, where the process 800 can display anoption to open an instructional module to the oncologist at the userinterface. Control then passes to block 818, where if the oncologistselects the option to open the instructional module, the process 800 canproceed to block 820. If the oncologist does not select the option toopen the instructional module, the process 800 can proceed to block 822.At block 820, the process 800 can display an instructional manual (i.e.,a user manual) to the oncologist. Control then passes to block 822,where the process 800 can display a FAQ menu as well as a suggestedpathways tutorial option to the oncologist at the user interface.Control then passes to block 824, where if the oncologist selects thesuggested pathways tutorial option, control passes to block 826. If theoncologist does not select the suggested pathways tutorial option, theprocess 800 ends.

At block 826, the process 800 can run at least one tutorial that caninclude tutorials about how to use the collaboration device 20 (e.g.,how to change the volume of the collaboration device 20) as well assuggest “first questions” the oncologist may want to ask thecollaboration device. In particular, tutorials related to suggestedquestions may instruct the oncologist on methods for querying thecollaboration device. After the oncologist practices with a number ofquestions, the collaboration device 20 can exit the tutorial and allowthe oncologist to ask questions independently. The tutorials can begenerated by recognizing the types of intents that the specificphysician may have and anticipating the questions based on variouscriteria (e.g., institution, areas of specialty, questions asked byother physicians that they are affiliated with, the patients’molecular/clinical data and their past order history, upcoming patientsbased on EMR scheduling integration, etc.). The tutorial can instructthe user visually and/or audibly about basic voice commands thecollaboration device can recognize such as “Volume Up,” “Volume Down,”“Start Pairing,” “Turn Off,” or other suitable voice commands.

Referring now to FIG. 32 , a screen shot 1100 of an interface for use bya system administrator for visually specifying system intents, intentparameters and answer formats for provider panel types that isconsistent with at least some aspects of the present disclosure isshown. As shown, a panel variable module 1104, an intent module 1108,and an answer module 1112 can be used by a user to specify intents. Itis noted that the modules can appear differently to make identificationeasier. For example, the modules 1104-1112 can be different shapes. Theintent module can have an intent (i.e., “Intent A”) that may require avariable input to answer. In this example, the variable is a panel type.The panel variable module 1104, which corresponds to a type of panel,can be linked to the intent module 1108, which can also be linked to theanswer module 1112 which can automatically fill in the answer based onthe panel variable module 1104. The user can drag and link the modules1104-1112 using a mouse or touchscreen to create the intent andassociated answer.

Referring now to FIG. 1 well as FIG. 33 , an intent extractionarchitecture 1150 that is consistent with at least some aspects of thepresent disclosure is shown. The intent extraction architecture 1150 caninclude an input module 1158 including a microphone and an output module1162 including a speaker that may be included in the collaborationdevice 20. A user 1154 can provide audible queries to the input module1158 and receive audible answers from the output module 1162. The inputmodule 1158 can process the audible query (e.g., perform textrecognition) and transmit a query 1166 to an intent matching module 1174included in the intent extraction architecture 1150. The intent matchingmodule 1174 can include an intent matching application such asDIALOGFLOW. The intent matching module 1174 can extract the intent fromthe query and transmit the query 1166 and the intent to a parameterextraction module 1178 included in the intent extraction architecture1150. The parameter extraction module 1178 can extract any parametersfrom the query 1166 that are relevant to the intent. The parameterextraction module 1178 can then communicate with an API module 1182and/or a database 1186 included in the intent extraction architecture1150 in order to extract information relevant to the extractedparameters and/or intent from the database 1186. The information can betransmitted to the intent matching module 1174. The intent matchingmodule 1174 can generate actionable data 1170 based on the informationfrom the database 1186. The intent matching module 1174 can thentransmit the actionable data 1170 to the output module 1162, which canoutput an audible answer based on the actionable data 1170 to the user1154.

Referring now to FIGS. 1, 28, and 33 as well as FIG. 34 , an exemplaryquestion and answer workflow 1200 that is consistent with at least someaspects of the present disclosure is shown. The workflow 1200 caninclude one or more collaboration devices 1204 (e.g., the collaborationdevice 20 and/or the mobile device 652), each including an input module1208 and an output module 1212. The input module 1208 may include atleast a portion of the components of the input module 1158, and theoutput module 1212 may include at least a portion of the components ofthe output module 1162. The input module 1208 can receive audiblequeries from an oncologist. The audible query can include a singlequestion that may be formulated from a series of prompts displayed onone of the collaboration devices 1204. The input module 1208 can outputthe audible query (which can include a raw audio file) to an agentmodule 1216 included in the workflow 1200. The agent module 1216 caninclude a number of natural language understanding (NLU) modules thattranslate text or spoken user requests into actions. The agent module1216 can translate the audible query into an action and transmit theaction to an intent matching module 1224 included in the workflow 1200.The intent matching module 1224 may be substantially the same as theintent matching module 1174. The intent matching module 1224 cancommunicate with a fulfillment module 1228 included in the workflow1200. The fulfillment module 1228 can include an intent-specific webhookfor agent look-up of business logic. The fulfillment module 1228 canreceive the intent from the intent matching module 1224 and communicatewith an API module 1232 included in the workflow 1200 to extractrelevant information from a database linked to the APImodule 1232. Thefulfillment module 1228 can then receive the relevant information fromthe API module 1232 and transmit the relevant information to the intentmatching module 1224. The intent matching module 1224 can then generatea response 1220 based on the relevant information and transmit theresponse 1220 to the output module 1212. The output module 1212 can thenvisually and/or audible output the response 1220.

Referring now to FIGS. 33 and 34 as well as FIG. 35 , an exemplaryconversation workflow 1250 that is consistent with at least some aspectsof the present disclosure is shown. A collaboration device 1254 (e.g.,the collaboration device 20) can receive an audible query. For example,the audible query can be “Hey ONE, how many patients have I sent toTEMPUS in the last 60 days with an identified PIK3CA mutation?” Thecollaboration device 1254 can transmit text extracted from the audiblequery to an intent matching module 1258, which can be substantially thesame as the intent matching module 1224. The intent matching module 1258can extract an intent associated with the audible query from the text.For example, the intent can be “one.patients.count.” The intent matchingmodule 1258 can transmit the text associated with the audible query andthe intent to an entities module 1262 that may be substantially the sameas the parameter extraction module 1178. The entities module 1262 candetermine one or more parameters based on the text. For example, theentities module 1262 can determine a status parameter (e.g., @status:Sequenced), a mutation parameter (e.g., @mutation: PIK3CA), and atimeframe parameter (e.g., @timeframe: 60). The entities module 1262 cantransmit the parameters, the text, and/or the intent to a fulfillmentmodule 1266 that formulates the parameters as a request to a databasethat includes that actual values of the parameters. The fulfillmentmodule 1266 transmits the request to an intent to database matchingmodule 1270 that extracts the requested values from a database. Theintent to database matching module 1270 can then output the requestedvalues to as response module 1274 that may be included in the intentmatching module 1258. The response module 1274 can generate a responseand transmit the response to the collaboration device 1254. For example,the response module 1274 can generate and transmit “There were 17patients with an identified PIK3CA mutation sent back to you in the last60 days.” The collaboration device 1254 can then audibly and/or visuallyoutput the response.

The methods and systems described above may be utilized in combinationwith or as part of a digital and laboratory health care platform that isgenerally targeted to medical care and research, and in particular,generating a molecular report as part of a targeted medical careprecision medicine treatment or research. It should be understood thatmany uses of the methods and systems described above, in combinationwith such a platform, are possible. One example of such a platform isdescribed in U.S. Pat. Publication No. 2021/0090694, titled “Data BasedCancer Research and Treatment Systems and Methods” (hereinafter “the‘694 publication”), which is incorporated herein by reference and in itsentirety for all purposes.

In some aspects, a physician or other individual may utilize acollaboration device, such as the collaboration device 20 or the mobiledevice 652, in connection with one or more expert treatment systemdatabases shown in FIG. 1 of the ‘694 publication. The collaborationdevice may operate on one or more micro-services operating as part of asystem services/applications/integration resources database, and themethods described herein may be executed as one or more systemorchestration modules/resources, operational applications, or analyticalapplications. At least some of the methods (e.g., microservices) can beimplemented as computer readable instructions that can be executed byone or more computational devices, such as the collaboration device 20,a server such as the first server 756, the second server 766, the thirdserver 776, the fourth server 772, the fifth server 788, the AI server14, or the collaboration server 12, and/or the administrator device 752.The one or more computational devices can be included in a systemdescribed above, such as the fifth system 750.

For example, an implementation of one or more embodiments of the methodsand systems as described above may include microservices included in adigital and laboratory health care platform that can audibly broadcastresponses to a physician in response to a query about a patient’smolecular report.

In some embodiments, a system can include a single microservice forexecuting and delivering the response to the query or may include aplurality of microservices, each microservice having a particular rolewhich together implement one or more of the embodiments above. In oneexample, a first microservice can include listening for a query from amicrophone of a collaboration device or otherwise receiving the queryfrom the user, identifying an intent associated with the query, andidentifying a data operation associated with the identified intent inorder to deliver a structured query having a data operation to beperformed on the patient’s molecular report to a second microservice forprocessing the query. Similarly, the second microservice may includeperforming the data operation on the patient’s molecular report in orderto generate response data, generating an audible response file from theresponse data, and providing that audible response data to thecollaboration so that the collaboration device may provide the audibleresponse to the physician in response to their query according to anembodiment, above.

The collaboration device may be utilized as a source for automated entryof the kind identified in FIG. 59 of the ‘694 publication. For example,the collaboration device may interact with an order intake server togenerate an order for a test. Where embodiments above are executed inone or more micro-services with or as part of a digital and laboratoryhealth care platform, one or more of such micro-services may be part ofan order management system that orchestrates the sequence of events asneeded at the appropriate time and in the appropriate order necessary toinstantiate embodiments above.

For example, continuing with the above first and second microservices,an order management system may notify the first microservice that anorder for an audible query has been received and is ready forprocessing. The first microservice may include executing and notifyingthe order management system once the delivery of a structured query isready for the second microservice. Furthermore, the order managementsystem may identify that execution parameters (prerequisites) for thesecond microservice are satisfied, including that the first microservicehas completed, and notify the second microservice that it may continueprocessing the order to provide the audible response to thecollaboration device according to an embodiment, above. While twomicroservices are utilized for illustrative purposes, queryidentification, intent identification, data operation association andexecutions, and audible response generation and delivery may be split upbetween any number of microservices for audibly broadcasting responsesto user based queries in accordance with performing embodiments herein.

In another example, the microservices included in a digital andlaboratory health care platform and capable of supporting audiblybroadcasting responses to a physician in response to a query about astatus of a patient’s molecular report can include identifying a statusof the progress of the generation of the patients report. The platformcan send a query to an order management system to request a currentstatus of the order for a respective patient. The other managementsystem may identify the last completed microservice which has broadcasta completion message and return the current stage of the patient’s orderand/or a time remaining until a report may be generated. Thecollaboration device may broadcast the current status as received fromthe order management system.

In some aspects, a physician or other individual may utilize acollaboration device in connection with one or more electronic documentabstraction services shown in FIG. 80 of the ‘694 publication. In someembodiments, the collaboration device can receive a verbal request tosummarize a portion or the whole of an electronic document. Theelectronic document may be identified and provided to an abstractionmicroservice for consumption for generating a structured data format ofthe information contained in the electronic document. The ab stractionmicroservice, or a subsequent microservice, may include identifyingwhich information of the information contained in the electronicdocument contains medically important data and generating an audibleresponse to the user’s query to summarize the document. In an example, asummary of a genetic sequencing report may identify the somatic variantsof the patient, the matched therapies which may be prescribed to thepatient, and potential clinical trials mentioned in the report. Inanother example, a summary of a patient’s clinical history may begenerated from the electronic health records and/or progress notesavailable to the microservices platform. A system (e.g., the fifthsystem 750) can generate an audible response to describe the patient’streatment history, family history, or other important medicalinformation contained in the medical record for playback to thephysician.

In some aspects, a physician or other individual may utilize acollaboration device (e.g., the collaboration device 20 or the mobiledevice 652) in connection with one or more electronic documentabstraction services shown in FIGS. 158-160 of the ‘694 publication. Thecollaboration device may receive a verbal request to summarize a portionor the whole of a physical document. The collaboration device mayrequest the physician to open a corresponding application on theirmobile device to capture the physical document in an electronic format,converting the physical document to an electronic document. Theelectronic document may then be provided to an abstraction microservicefor consumption for generating a structured data format of theinformation contained in the electronic document. Summarization andaudible response generation may be performed as described above withrespect to another aspect.

In some aspects, a physician or other individual may utilize acollaboration device (e.g., the collaboration device 20 or the mobiledevice 652) in connection with one or more prediction engine servicesshown in FIG. 204 of the ‘694 publication. The collaboration device mayreceive a verbal request to predict an outcome for a patient withrespect to a specific target outcome and within a specified time period.A query identifying the patient, the outcome, and the time period may besent to a prediction engine to generate a prediction. In anotherembodiment, predictions may be precomputed and stored in a patientprediction database for retrieval. An audible query response includingthe prediction may be generated provided to the collaboration device forplayback to the physician. Queryable predictions may include targetssuch as odds of progression-free survival, death, metastasis, occurrenceof disease progression states, or other predictable outcomes and timeperiods measured in days, weeks, months, or years.

In another aspect, a pathologist or other individual may utilize acollaboration device (e.g., the collaboration device 20 or the mobiledevice 652) in connection with one or more cell-type profiling servicesshown in FIG. 244 of the ‘694 publication. The collaboration device mayreceive a verbal request to identify cell and tissue types present in anH&E or IHC slide from a tumor next-generation sequencing reportgenerated from the slide or a slide proximate to the sequenced slide. Acell-type profiling service may identify the cell-types present in theslide and generate an audible query response to provide the identifiedcell-types to the pathologist. In another aspect, the collaborationdevice may receive a verbal request to identify an unknown tumor originof a tumor tissue present in a slide. The cell-type profiling servicemay identify the tumorous cell-types present in the slide as originatingfrom an organ of the body and identify that the cell-types likelyrepresent a metastasis from an organ to the site where the tumor tissuewas biopsied. Consistent with the above aspects, an audible queryresponse may be generated to provide the origin of the identifiedtumorous cell-types to the physician.

In some aspects, a physician or other individual may utilize acollaboration device (e.g., the collaboration device 20 or the mobiledevice 652) in connection with one or more tissue segmentation servicesshown in FIGS. 253 and 261 of the ‘804 application. The collaborationdevice may receive a verbal request to summarize classification of aportion or the whole of a digital representation of an H&E or IHC slide.The digital slide may be identified and provided to a tissuesegmentation microservice for consumption, classification of the tissuepresent, and summarization. The segmentation microservice, or asubsequent microservice, may include identifying the cell/tissue typesand proportions present in the digital slide and generating an audibleresponse to the user’s query to summarize the types and respectiveproportions to the physician.

The digital and laboratory health care platform further includes one ormore insight engines shown in FIG. 272 . Exemplary insight engines mayinclude a tumor of unknown origin engine, a human leukocyte antigen(HLA) loss of homozygosity (LOH) engine, a tumor mutational burden (TMB)engine, a PD-L1 status engine, a homologous recombination deficiency(HRD) engine, a cellular pathway activation report engine, an immuneinfiltration engine, a microsatellite instability engine, a pathogeninfection status engine, and so forth as described with respect to FIG.189 , 199-200, and 266-270 of the ‘694 publication. In an aspect, aphysician may query the collaboration device as to the patient’s statusfor any diagnosis of the patient as to an insight engine such as HLALOH, TMB, PD-L1, HRD, active pathway, or other insight status. Thecollaboration device may identify an insight engine query by keywordcontent matching one of the existing insight engines and generate acorresponding database query to retrieve the related status associatedwith the patient. The related status of the patient may then be providedas part of an audible response to the physician by the collaborationdevice. In some examples, an audible response may identify the sourcediagnostic testing which provided the baseline for the insight as wellas the date of collection. For example, an audible response may include,“Patient has been identified as TMB High as a result of next generationsequencing of the patient’s breast tumor on Jan. 21, 2019.”

When the digital and laboratory health care platform further includes amolecular report generation engine, the methods and systems describedabove may be utilized to create a summary report of a patient’s geneticprofile and the results of one or more insight engines for presentationto a physician. For instance, the report may provide to the physicianinformation about the extent to which the specimen that was sequencedcontained tumor or normal tissue from a first organ, a second organ, athird organ, and so forth. For example, the report may provide a geneticprofile for each of the tissue types, tumors, or organs in the specimen.The genetic profile may represent genetic sequences present in thetissue type, tumor, or organ and may include variants, expressionlevels, information about gene products, or other information that couldbe derived from genetic analysis of a tissue, tumor, or organ via agenetic analyzer. The report may further include therapies and/orclinical trials matched based on a portion or all of the genetic profileor insight engine findings and summaries shown in FIGS. 271 and 302 ofthe ‘694 publication. A physician, or other individual, may query thecollaborative device as to the therapies or clinical trials the patientmay qualify for. In one example, the collaboration device may referencea database having previously stored the patient’s potential therapiesand clinical trials. In another example, the collaboration device mayinitiate a new identification of potential therapies and/or clinicaltrials based upon the query received. A microservice may includedetermining the patient’s eligibility based on all current patientinformation, identifying the closest matches, and generating, or causinganother microservice to generate, an audible response with the closestmatching therapies or clinical trials. The collaboration device mayreceive and broadcast the audible response to the physician.

It should be understood that the examples given above are illustrativeand do not limit the uses of the systems and methods described herein incombination with a digital and laboratory health care platform.

Referring now to FIG. 36 , a process 1300 that is consistent with atleast some aspects of the present disclosure is shown that provides anaudible response to an oncologist using at least one microservice and/orengine is shown. The process 1300 can be executed (i.e., performed) inconjunction with (e.g., in parallel with) the process 450 in order toprovide relevant information to the oncologist based on audible queriesenunciated by the oncologist. Furthermore, the process 1300 can utilizedin combination with or as part of a digital and laboratory health careplatform, such as the platform is described in the ‘804 application. Thedigital and laboratory health care platform can include one or moremicroservices and/or may operate on one or more microservices. In otherwords, the digital and laboratory health care platform may beimplemented using microprocesses. In some embodiments, the digital andlaboratory health care platform can generate a molecular report as partof a targeted medical care precision medicine treatment.

At process block 1302, the process 1300 can receive an audible queryfrom an oncologist via a collaboration device microphone. In someembodiments, the collaboration device can include the collaborationdevice 20. In some embodiments, the collaboration device can include themobile device 652. In some aspects, the audible query can be a verbalrequest to summarize a portion or the whole of an electronic documentrelated to the patient (e.g., “Summarize the first page of the molecularreport for Dwayne Holder”), a verbal request about the status of thepatient’s molecular report, a verbal request to predict an outcome for apatient with respect to a specific target outcome and within a specifiedtime period (e.g., “Predict one year survival for Dwayne Holder”), averbal request to identify cell and tissue types present in an H&E orIHC slide from a tumor next-generation sequencing report generated fromthe slide or a slide proximate to the sequenced slide, a verbal requestto summarize a classification of a portion or the whole of a digitalrepresentation of an H&E or IHC slide, or another suitable verbalrequest.

At block 1304 the process 1300 can identify at least one intentassociated with the audible query. For example, if the audible query is“Summarize the first page of the molecular report for Dwayne Holder,”the intent may be “Summarize a portion of this document” and specificquery parameters may include “first page” and “molecular report forDwayne Holder” where the underlined portion and document in the generalquery are populated with “first page” and “molecular report for DwayneHolder” to generate a specific query intent. As another example, if theaudible query is “What is the estimated one year survival for DwayneHolder?” the intent may be “Estimate probability of outcome in timeperiod for patient” and specific query parameters may include“survival,” “one year,” and “Dwayne Holder” for the underlined outcome,time period, and patient respectively.

At block 1306 the process 1300 can identify at least one data operationassociated with the specific intent. The at least one data operation caninclude providing the at least one intent to at least one microserviceand/or engine. For example, if the intent is “Summarize a portion ofthis document,” the process 1300 can provide the portion and documentparameters to an abstraction microservice. As another example, if theintent is “Estimate probability of outcome in time period for patient,”the process 1300 can provide the outcome, time period, and patientparameters to a prediction engine. Thus, the process 1300 can beperformed in conjunction with one or more microservices. In someembodiments, the process 1300 can be performed in conjunction with oneor more microservices of an order management system. In theseembodiments, the process 1300 can access or report on a status of anorder as is flows through various databases and/or servers. In someembodiments, the process 1300 can be performed in conjunction with oneor more microservices of a medical document abstraction system. In theseembodiments, the process 1300 can access or report on contents ofphysical or electronic medical documents. In some embodiments, theprocess 1300 can be performed in conjunction with one or moremicroservices of a mobile device application. In these embodiments, theprocess 1300 can access or report on the status of a patient, physicalor electronic documents, or other application interfaced data. In someembodiments, the process 1300 can be performed in conjunction with oneor more microservices of a prediction engine. In these embodiments, theprocess 1300 can access precomputed predictions of a patient or requeston demand predictions to be generated regarding the status of a patient.In some embodiments, the process 1300 can be performed in conjunctionwith one or more microservices of a service such as a cell-typeprofiling service. In these embodiments, the process 1300 can accessprecomputed cell-type predictions from sequencing or trigger computationon a given raw sequencing data or digital image. In some embodiments,the process 1300 can be performed in conjunction with a variant callingengine to provide information to a query involving variants. In theseembodiments, the process 1300 can access a database (e.g., a TEMPUSdatabase) to provide sequencing results. In some embodiments, theprocess 1300 can be performed in conjunction with an insight engine. Inthese embodiments, the process 1300 can access a database (e.g., aTEMPUS database) to provide advanced analytic results for a particularinsight test [TUO, HLA LOH, TMB, PD-L1, HRD, active pathway, or otherinsight status]). In some embodiments, the process 1300 can be performedin conjunction with a therapy matching engine. In these embodiments, theprocess 1300 can access therapies which are relevant to the patient. Insome embodiments, the process 1300 can be performed in conjunction witha clinical trial matching engine. In these embodiments, the process 1300can access clinical trials which are relevant to the patient.

At block 1308, the process 1300 can execute the at least one dataoperation using at least one microservice and/or at least one engine.The at least one microservice and/or at least one engine can generateresponse data based on the parameters identified at block 1304.

At block 1310, the process 1300 can formulate a suitable audio responsefile based on the response data received from the at least onemicroservice and/or the at least one engine. For example, theabstraction microservice can identify which information of theinformation contained in the electronic document contains medicallyimportant data, and include the medically important data in the responsedata provided to the process 1300. The process 1300 can then include atleast a portion of the medically important data in the audio responsefile. As another example, the prediction engine can generate anestimated survival figure (e.g., a percentage), and include theestimated survival figure in the response data provided to the process1300.

At block 1312, the process 1300 can broadcast an audible response to theoncologist. More specifically, the process 1300 can cause the audibleresponse to be output at speakers included in the collaboration device.The audible response can be the result of the collaboration deviceactuating the speakers based on the audio response file.

Appendix D includes an exemplary set of questions and answers that anoncologist may voice to a disclosed collaboration device and that thedevice may return in response. While clearly not exhaustive, theexemplary questions and answers give a sense of the power of the systemand the complexity of the types of queries that the system can handle.

Table 2 below includes an exemplary set of questions and answers that anoncologist may voice to a disclosed collaboration device and that thedevice may return in response. While clearly not exhaustive, theexemplary questions and answers give a sense of the power of the systemand the complexity of the types of queries that the system can handle.

TABLE 2 Category Question Answer Answer for negative Answer for NullReport Content Where was Dwayne Holder’s tumor sample collected from?Dwayne Holder’s tumor sample was collected from a lung biopsy. TEMPUSdoes not have information on where Dwayne Holder’s tumor sample wascollected from. Please check the TEMPUS portal for information aboutDwayne Holder’s tumor sample. Report Content What is the tumorpercentage of Dwayne Holder’s tumor sample submitted to TEMPUS forsequencing? The tumor percentage of Dwayne Holder’s sample was 40%.TEMPUS does not currently have information on Dwayne Holder’s tumorpercentage. Please check the TEMPUS portal for information about DwayneHolder’s tumor percentage. Report Content What is Dwayne Holder’sdiagnosis? Dwayne Holder has a Pancreatic Ductal Adenocarcinoma. TEMPUSdoes not currently have information on Dwayne Holder’s diagnosis. Pleasecheck the TEMPUS portal for information about Dwayne Holder’s diagnosis.Report Content What is Dwayne Holder’s Date Of Birth? Dwayne Holder’sdate of birth is August 8th, 1971. TEMPUS does not currently haveinformation on Dwayne Holder’s date of birth. Please check the TEMPUSportal for information on Dwayne Holder’s date of birth. Report ContentWhat is Dwayne Holder’s TMB? Dwayne Holder’s TMB was found to be in the79th percentile. Dwayne Holder’s TEMPUS xT report did not have a TMBassociated with it. Please check the TEMPUS portal for information onDwayne Holder’s TMB. Report Content What genes does Dwayne Holder have agermline mutation in? Dwayne Holder was found to have a germlinemutation in his BRCA2 gene during sequencing. TEMPUS did not find anygermline mutations for Dwayne Holder during sequencing. TEMPUS did notreceive a normal sample for Dwayne Holder, therefore no germlinemutations were found. Report Content How many fusions does Dwayne Holderhave? Dwayne Holder was found to have 1 clinically validated fusion.Dwayne Holder did not have any fusions found during sequencing. Pleasecheck the TEMPUS portal for information on Dwayne Holder’s mutations.Report Content Has Dwayne Holder received any previous lines of therapy?Dwayne Holder has previously received folfirinox and olaparib. TEMPUSdoes not currently have any information about any previous therapies forDwayne Holder. Please check the TEMPUS portal for information on DwayneHolder’s previous lines of therapy. Report Content What report editionis Dwayne Holder’s report? Dwayne Holder has the original reportedition. Please check the TEMPUS portal for information on DwayneHolder’s report edition Please check the TEMPUS portal for informationon Dwayne Holder’s report edition Report Content What was DwayneHolder’s date of diagnosis? Dwayne Holder’s diagnosis date is July 27th,2018. TEMPUS does not have information on Dwayne Holder’s date ofdiagnosis. Please check the TEMPUS portal for information on DwayneHolder’s diagnosis date. Report Content Have there been any amendmentsto Dwayne Holder’s report? Yes, there has been an amendment to DwayneHolder’s report. No, there has not been an amendment to Dwayne Holder’sreport. Please check the TEMPUS portal for information on DwayneHolder’s reports, including any amendments made. Report Content IsDwayne Holder MSI high? Yes, Dwayne Holder is MSI high. Dwayne Holder isMSI stable. Please check the TEMPUS portal for information on DwayneHolder’s MSI status. Report Content What is Dwayne Holder’s mutationsper Megabase? Dwayne Holder was found to have 2.7 mutations per Megabaseduring sequencing. Dwayne Holder does not have any informationassociated with TMB on his report. Please check the TEMPUS portal forinformation on Dwayne Holder’s TMB. Report Content What FDA approved, onlabel drugs does TEMPUS recommend for Dwayne Holder? TEMPUS does notrecommend drugs or other therapies. Selection of therapies should beperformed through the physician’s judgment. TEMPUS does not recommenddrugs or other therapies. Selection of therapies should be used by thephysician’s judgment. TEMPUS does not recommend drugs or othertherapies. Selection of therapies should be performed through thephysician’s judgment. Report Content What drugs have been approved bythe FDA for patients who have mutations like Dwyane Holder? KEYTRUDA hasbeen approved by the FDA for patients who are MSI high and meet theother labeling criteria. TEMPUS does not have any FDA approved drugsassociated with patients that have mutations like Dwayne Holder’s.Please check the TEMPUS portal for more information on Dwayne Holder’sstaging information. Report Content What FDA approved, off label drugsdoes TEMPUS recommend for TEMPUS does not recommend drugs or othertherapies. Selection of therapies should be used by the TEMPUS does notrecommend drugs or other therapies. Selection of therapies should beperformed through TEMPUS does not recommend drugs or other therapies.Selection of therapies should be performed through the physician’sjudgment. Dwayne Holder? physician’s judgment (see disclaimer text) thephysician’s judgment. Report Content What FDA approved therapies forother indications exist for Dwayne Holder? Based on Dwayne Holder’smutations, olaparib, niraparib, rucaparib, talazoparib, and palbociclibare FDA approved therapies for other indications. TEMPUS does not haveany FDA approved drugs associated with patients that have mutations likeDwayne Holder’s. Please check the TEMPUS portal for more information onDwayne Holder’s staging information. Report Content What investigationaldrugs does TEMPUS recommend for Dwayne Holder? TEMPUS does not recommenddrugs or other therapies. Selection of therapies should be used by thephysician’s judgment (see disclaimer text) TEMPUS does not recommenddrugs or other therapies. Selection of therapies should be performedthrough the physician’s judgment. TEMPUS does not recommend drugs orother therapies. Selection of therapies should be performed through thephysician’s judgment. Report Content What investigational drugs areassociated with patients like Dwayne Holder? A preclinical study foundVerteporfin and LY-3009120, an investigational drug regimen, to have agood response in KRAS G.12V mutations. There is not much researchcurrently surrounding investigational drugs associated with patientsthat have mutations like Dwayne Holder’s. Please check the TEMPUS portalfor more information on Dwayne Holder’s staging information. ReportContent What AJCC staging does Dwayne Holder have? Dwayne Holder has astage T3 N1 M1 tumor. TEMPUS does not have information about DwayneHolder’s AJCC staging. Please check the TEMPUS portal for moreinformation on Dwayne Holder’s staging information. Report Content Howmany mutations did TEMPUS find for Dwayne Holder? Dwayne Holder wasfound to have 5 mutations during sequencing. TEMPUS did not find anymutations for Dwayne Holder during sequencing. Please check the TEMPUSportal for more information on Dwayne Holder’s mutations. Report ContentWhat somatic mutations does Dwayne Holder have? Dwayne Holder was foundto have a CDKN2A, KRAS, and a BRCA2 somatic mutation during sequencing.TEMPUS did not find any somatic mutations for Dwayne Holder duringsequencing. Please check the TEMPUS portal for more information onDwayne Holder’s mutations. Report Content What can I give Dwayne Holderfor his KRAS mutation? Verteporfin may be a treatment option givenDwayne Holder’s KRAS TEMPUS finds that Dwayne Holder’s KRAS mutation issomatic and Please check the TEMPUS portal for more information onDwayne Holder’s mutations. mutation. Refer to the complete report foradditional details. biologically relevant, though there are no therapiescurrently associated with this specific mutation. Report Content HasDwayne Holder had any biopsies? Dwayne Holder has had a lung biopsy.TEMPUS does not have a record of any previous biopsies for DwayneHolder, though this information may not have been shared with TEMPUS.Please check the TEMPUS portal for more information on Dwayne Holder’sclinical history. Report Content When was Dwayne Holder’s lung biopsy?Dwayne Holder’s biopsy was performed on February 28th, 2019. TEMPUS doesnot have a record of any previous biopsies for Dwayne Holder, Pleasecheck the TEMPUS portal for more information on Dwayne Holder’s clinicalhistory. though this information may not have been shared with TEMPUS.Report Content Has Dwayne Holder taken a PARP inhibitor before? DwayneHolder has previously taken a PARP inhibitor. Dwayne Holder has neverreceived a PARP inhibitor, though this information may not have beenshared with TEMPUS. Please check the TEMPUS portal for more informationon Dwayne Holder’s clinical history. Report Content What drug classesdoes TEMPUS associate with Dwayne Holder’s mutations? Based on DwayneHolder’s mutations, an EGFR Inhibitor, Pyrimidine Analog, PARPInhibitor, CDK4/6 inhibitor, YAP Inhibitor, or a Pan-RAF Inhibitor aredrug classes that are investigational or FDA approved for otherindications. TEMPUS does not currently have any associated therapies forDwayne Holder’s mutations. Please check the TEMPUS portal for moreinformation on Dwayne Holder’s mutations and associated therapies.Report Content What drug classes does TEMPUS recommend for patients withmutations like TEMPUS does not recommend drugs or other therapies.Selection of therapies should be used by the TEMPUS does not recommenddrugs or other therapies. Selection of therapies should be used by theTEMPUS does not recommend drugs or other therapies. Selection oftherapies should be used by the physician’s Dwayne Holder? physician’sjudgment (see disclaimer text) physician’s judgment (see disclaimertext) judgment (see disclaimer text) Report Content What FDA approveddrugs does TEMPUS recommend for Dwayne Holder? TEMPUS does not recommenddrugs or other therapies. Selection of therapies should be used by thephysician’s judgment (see disclaimer text) TEMPUS does not recommenddrugs or other therapies. Selection of therapies should be used by thephysician’s judgment (see disclaimer text) TEMPUS does not recommenddrugs or other therapies. Selection of therapies should be used by thephysician’s judgment (see disclaimer text) Report Content What FDAapproved drugs does TEMPUS associate with patients who have mutationslike Dwayne Holder? Based on Dwayne Holder’s mutations, olaparib,rucaparib, niraparib, talazoparib, and palbociclib are FDA approvedtherapies. There currently are no FDA approved drugs associated withDwayne Holder’s mutations, though there may be investigational drugs.Please check the TEMPUS portal for more information on Dwayne Holder’smutations and associated therapies. Report Content Has Dwayne Holdertaken olaparib before? Dwayne Holder has taken olaparib before. DwayneHolder has never received olaparib, though this information may not havebeen shared with TEMPUS. Please check the TEMPUS portal for moreinformation on Dwayne Holder’s clinical history. Report Content Isgemcitabine with erlotinib an option for Dwayne Holder? Dwayne Holderhas an X mutation, which was associated with a positive response togemcitabine and erlotinib in a preclinical trial. Gemcitabine witherlotinib is associated with a resistance in patients with KRASmutations like Dwayne Holder. Please check the TEMPUS portal for moreinformation on Dwayne Holder’s mutations and associated therapies.Report Content What drugs does TEMPUS recommend to treat Dwayne Holder?TEMPUS does not recommend drugs or other therapies. Selection oftherapies should be used by the physician’s judgment (see disclaimertext) TEMPUS does not recommend drugs or other therapies. Selection oftherapies should be used by the physician’s judgment (see disclaimertext) TEMPUS does not recommend drugs or other therapies. Selection oftherapies should be used by the physician’s judgment (see disclaimertext) Report Content What drugs does TEMPUS associate with patients whohave mutations Based on Dwayne Holder’s mutations, olaparib, rucaparib,niraparib, talapozib, palbociclib, verteporfin, and TEMPUS does notrecommend drugs or other therapies. Selection of therapies should beused by the Please check the TEMPUS portal for more information onDwayne Holder’s mutations and associated therapies. like DwayneHolder’s? LY3009120 are drugs that are investigational or FDA approvedfor other indications. physician’s judgment (see disclaimer text) ReportContent Is erlotinib with gemcitabine FDA approved? Erlotinib andgemcitabine is an FDA approved regimen in some instances. Erlotinib andgemcitabine is not currently an FDA approved regimen. Please check theTEMPUS portal for more information on Dwayne Holder’s mutations andassociated therapies. Report Content Is the erlotinib with gemcitabinetherapy on Dwayne Holder’s report an on-label use of the drugs?Erlotinib with gemcitabine is an FDA approved regimen in some instances,though given Dwayne Holder’s KRAS mutation, he may be resistance to thistherapy. Erlotinib and gemcitabine is not currently an FDA approvedregimen. Please check the TEMPUS portal for more information on DwayneHolder’s mutations and associated therapies. Report Content What type ofmutation is Dwayne Holder’s KRAS mutation? Dwayne Holder has a gain offunction KRAS mutation. Dwayne Holder was not found to have a KRASmutation. Please check the TEMPUS portal for more information on DwayneHolder’s mutations and associated therapies. Report Content What genesdoes Dwayne Holder have a germline mutation in? Dwayne Holder has aBRCA2 germline mutation. Dwayne Holder was not found to have anygermline mutations. Please check the TEMPUS portal for more informationon Dwayne Holder’s mutations. Report Content What grade is DwayneHolder’s tumor? Dwayne Holder has a grade 3 tumor. TEMPUS does not haveinformation about Dwayne Holder’s histologic grade. Please check theTEMPUS portal for more information on Dwayne Holder’s histologic grade.Report Content What stage is Dwayne Holder? Dwayne Holder is stage 4.TEMPUS does not have information about Dwayne Holder’s stage. Pleasecheck the TEMPUS portal for more information on Dwayne Holder’s stage.Report Content Did Dwayne Holder progress on any of his previoustherapies? According to the clinical records provided to TEMPUS, DwayneHolder progressed while on folfirinox. TEMPUS does not have a record ofany progressions for Dwayne Holder in his clinical history, though thisinformation may not Please check the TEMPUS portal for more informationon Dwayne Holder’s clinical history. have been shared with TEMPUS.Report Content Did Dwayne Holder have a response to Olaparib? Accordingto the clinical records provided to TEMPUS, Dwayne Holder had a partialresponse to olaparib. TEMPUS does not have a record of any responses toolaparib for Dwayne Holder in his clinical history, though thisinformation may not have been shared with TEMPUS. Please check theTEMPUS portal for more information on Dwayne Holder’s clinical history.Report Content What clinical trials on Dwayne Holder’s report arerelated to his MSI status? The Keytruda clinical trial on DwayneHolder’s report is related to his MSI status. TEMPUS did not include anyclinical trials on Dwayne Holder’s report that are associated with hisMSI status, though he does have 6 other trials listed that may be a fit.Please check the TEMPUS portal for more information on Dwayne Holder’sclinical matched trials. Report Content What is the clinicalsignificance of Dwayne Holder’s germline BRCA2 mutation? Dwayne Holder’sBRCA2 germline mutation is classified as a pathogenic variant. DwayneHolder was not found to have a germline BRCA2 mutation. Please check theTEMPUS clinical portal for more information on Dwayne Holder’smutations. Report Content Are there any diseases associated with DwayneHolder’s BRCA2 mutation? Dwayne Holder’s BRCA2 germline mutation isassociated with Hereditary breast and ovarian cancer. TEMPUS does nothave any diseases associated with Dwayne Holder’s germline BRCA2mutation at this time. Please check the TEMPUS clinical portal for moreinformation on Dwayne Holder’s mutations. Order Status What is thestatus of Dwayne Holder’s order? Dwayne Holder’s order is currently inprogress. There is no order status associated with Dwayne Holder’sorder. Please check the TEMPUS portal to find more information aboutDwayne Holder’s order status. Order Status When was Dwayne Holder’sorder placed? Dwayne Holder’s order was placed on Jul. 16, 2019. Thereis no date associated with Dwayne Holder’s order. Please check theTEMPUS portal to find more information about Dwayne Holder’s ordersubmission date. Order Status What is the current status of DwayneHolder’s DNA order? Dwayne Holder’s DNA order is currently complete. Thestatus of Dwayne Holder’s DNA order can not be retrieved at this time.Please check the TEMPUS portal for information regarding the status ofDwayne Holder’s DNA order. Order Status What is the current status ofDwayne Holder’s IHC MMR order? Dwayne Holder’s IHC MMR order iscurrently delayed. The status of Dwayne Holder’s IHC MMR order can notbe retrieved at this time. Please check the TEMPUS portal forinformation regarding the status of Dwayne Holder’s IHC MMR order. OrderStatus What is the current status of Dwayne Holder’s PD-L1 SP142 order?Dwayne Holder’s PD-L1 SP142 order is currently in progress. The statusof Dwayne Holder’s PD-L1 SP142 order can not be retrieved at this time.Please check the TEMPUS portal for information regarding the status ofDwayne Holder’s PD-L1 SP142 order. Order Status What orders arecurrently in progress for Dwayne Holder? There are 2 orders currently inprogress for Dwayne Holder. Dwayne Holder’s in progress orders can notbe retrieved at this time. Please check the TEMPUS portal forinformation regarding Dwayne Holder’s in progress orders. Order StatusWhat orders are currently delayed for Dwayne Holder? There are 3 orderscurrently delayed for Dwayne Holder. Dwayne Holder’s delayed orders cannot be retrieved at this time. Please check the TEMPUS portal forinformation regarding Dwayne Holder’s delayed orders. Order Status Whatorders are currently complete for Dwayne Holder? There are 2 orderscurrently complete for Dwayne Holder. Dwayne Holder’s completed orderscannot be retrieved at this time. Please check the TEMPUS portal forinformation regarding Dwayne Holder’s in progress orders. Order StatusWhen did TEMPUS receive Dwayne Holder’s tumor sample? TEMPUS receivedDwayne Holder’s tumor sample on July 20th, 2019. TEMPUS has not yetreceived Dwayne Holder’s tumor sample. Please check the TEMPUS portalfor information regarding Dwayne Holder’s orders. Order Status When didTEMPUS receive Dwayne Holder’s normal sample? TEMPUS received DwayneHolder’s normal sample on July 21st, 2019. TEMPUS has not yet receivedDwayne Holder’s normal sample. Please check the TEMPUS portal forinformation regarding Dwayne Holder’s orders. Order Status When wasDwayne Holder’s DNA panel ordered? Dwayne Holder’s DNA panel was orderedon July 16th, 2019. TEMPUS has not received a DNA order for DwayneHolder. Please check the TEMPUS portal for information regarding DwayneHolder’s DNA orders. Clinical Reports 101 What type of database doesTEMPUS use for its clinical science information? TEMPUS uses a databasecalled the “Knowledge Database,” or KDB, to store its clinical scienceinformation. Could you repeat your question? Clinical Reports 101 Howdoes TEMPUS classify variants? TEMPUS has adapted standards from theACMG, AMP, ASCO and other working-groups to use for somatic and germlinevariant classification, in addition to TEMPUS uses a weightedclassification scheme for classifying variants. Molecular Platform is aproprietary internally-built program that utilizes a series ofalgorithms that incorporate internal and external databases and thelatest science to classify variants. Could you repeat your question?Clinical Reports 101 What variants does TEMPUS list on its reports?TEMPUS lists pathogenic, likely pathogenic, and variants of unknownsignificance on its reports. Could you repeat your question? ClinicalReports 101 What groups are variants classified under? Variants areclassified as pathogenic, likely pathogenic, variant of unknownsignificance, likely Could you repeat your question? benign, or benignvariants. Clinical Reports 101 Does TEMPUS include therapies for VUS’?TEMPUS includes therapies for pathogenic and likely pathogenic variantsonly. Could you repeat your question? Clinical Reports 101 Does TEMPUSinclude therapies for likely pathogenic variants? TEMPUS does includetherapies on its clinical reports for likely pathogenic variants. Couldyou repeat your question? Clinical Reports 101 What genes are listedunder biologically relevant? Somatic pathogenic and likely pathogenicvariants without therapies are “Biologically Relevant.” Could you repeatyour question? Clinical Reports 101 How does TEMPUS classify a gene as“Biologically Relevant”? TEMPUS classifies somatic pathogenic and likelypathogenic variants without associated therapies as “BiologicallyRelevant.” Could you repeat your question? Clinical Reports 101 Aregermline mutations reported for all patients? Germline mutations arereported for for tumor-normal consenting patients. Could you repeat yourquestion? Clinical Reports 101 Can germline mutations be reported forpatients that did not have a normal sample? Germline mutations are onlyreported for for tumor-normal consenting patients. Could you repeat yourquestion? Clinical Reports 101 What is the benefit of a tumor normalsample? TEMPUS -normal samples allows for clearly defines somaticmutations and can detect medically actionable germline variants. Couldyou repeat your question? Clinical Reports 101 How can TEMPUSdistinguish from a somatic or germline mutation? A tumor-normal sampleallows for clearly defines somatic mutations and can detect medicallyactionable germline variants. Could you repeat your question? ClinicalReports 102 Does TEMPUS report on incidental germline findings? Inaddition to somatic variants, TEMPUS reports Incidental GermlineVariants with patient consent. These are inherited variants in genesthat have been previously associated with an increased risk for certaintypes of cancer, or other medically actionable disorders recommended bythe American College of Medical Genetics and Genomics. Could you repeatthe type of mutation? Clinical Reports 103 How does TEMPUS calculateTMB? TMB is calculated as the number of all protein-altering mutationsper million base-pairs of DNA covered by the TEMPUS panel. For example,non-synonymous mutations are normalized to 2.4Mb, the size of the xTpanel. Could you repeat your question? Clinical Reports 103 How doesTEMPUS calculate MSI? TEMPUS measures MSI status through DNA sequencingand also offers clinical MMR protein IHC. Could you repeat yourquestion? Clinical Reports 101 What does Levels of Evidence mean on aTEMPUS report? The clinical evidence on a TEMPUS report integratesclinical expertise, patient medical information, and the best availablepeer-reviewed evidence, including publications, case studies,guidelines, and others into the decision making process for patientcare. Could you repeat your question? Clinical Reports 101 What does“Consensus” mean on a TEMPUS report? Consensus evidence on a TEMPUSreport refers to NCCN guideline and/or standard of care. Could yourepeat your question? Clinical Reports 101 What types of levels ofevidence are used on a TEMPUS report? The levels of evidence on a TEMPUSreport include Consensus, Clinical Research, Case Study, and PreClinicalevidence. Could you repeat your question? Clinical Reports 101 What does“Case Study” mean on a TEMPUS report? Case Study evidence on a TEMPUSreport refers to a case study or small patient cohort. Could you repeatyour question? Clinical Reports 101 What does “Preclinical” mean on aTEMPUS report? PreClinical evidence on a TEMPUS report refers topatient-derived xenograft modeling, cell lines, mouse models, ororganoid modeling studies. Could you repeat your question? ClinicalReports 101 What does Immune Infiltration mean on a The immuneinfiltration estimate on a TEMPUS report is meant for Could you repeatyour question? TEMPUS report? experienced physicians to have a startingpoint to examine the tumor-immune microenvironment. It estimates theproportions between B-, CD4+ T-, CD8+ T-, macrophages, and NK cells inthe immune cell infiltrate and does not currently consider any othertype of immune cell. This information is only for research use. ClinicalReports 101 What is Immune Infiltration of a tumor? Immune cells thatinfiltrate into the tumor can affect the response to immunotherapy. Theimmune infiltration module uses the gene expression patterns ofdifferent immune cell types to predict their relative abundance withinthe tumor. Higher levels of infiltrating immune cells, specificallycytotoxic CD8+ T cells, are associated with better responses tocheckpoint inhibition. Could you repeat your question? Clinical Reports101 What does HLA typing mean? HLA typing provides a patient’s genotypefor the three HLA class 1 genes. There are thousands of Could you repeatyour question? different alleles for each HLA gene, and each alleledisplays a different subset of peptides to the immune system. ClinicalReports 101 What does Neoantigen prediction mean on a TEMPUS report?Neoantigen prediction module leverages HLA typing to provide additionalcontext for the results of the TMB metric. Neoantigenic mutations aremutations resulting in a protein change in which from the resultingpeptide fragment is predicted to bind to the patient’s uniquecombination of HLA alleles. These mutations are more likely to bevisible to the immune system and to be the target of immune responses.Could you repeat your question? Clinical Reports 101 What isimmunotherapy resistance risk? Immunotherapy resistance risk refers tocertain gene alterations that have been associated with resistance toimmunotherapy regimens. Could you repeat your question? Clinical Reports101 What is a variant allele fraction? A Variant Allele Fraction is theproportion of variant reads for a given mutation. This represents thepercentage of tumor Could you repeat the term? cells that harbor aspecific mutation. Clinical Reports 101 What does potentially actionablemean on a TEMPUS report? Potentially actionable on a TEMPUS reportrefers to protein-altering variants with an associated therapy. Couldyou repeat the term?

Thus, the invention is to cover all modifications, equivalents, andalternatives falling within the spirit and scope of the invention asdefined by the following appended claims.

To apprise the public of the scope of this invention, the followingclaims are made:

1. A method of providing responses to a user based on user queries abouta specific patient’s molecular report information, the method for usewith a collaboration device that includes a processor, the methodcomprising: (i) storing molecular report information for a plurality ofpatients in a system database; (ii) receiving a query answerable in partby reference to a genomic biomarker from the user; (iii) identifying atleast one intent associated with the query, the at least one intentincluding at least one qualifying entity, the at least one intentidentified by a machine learning module recognizing the query assufficiently corresponding to an intent phrase provided to or generatedby the machine learning module; (iv) identifying at least one parametervalue associated with the entity in the query corresponding to the atleast one qualifying entity; (v) identifying at least one first dataoperation associated with the at least one intent, the at least onefirst data operation associated with the at least one qualifying entity;(vi) identifying at least one second data operation associated with thegenomic biomarker; (vii) executing (a) at least one of the identified atleast one first data operations and (b) at least one of the identifiedat least one second data operations on a first set of data in the systemdatabase, the first set of data comprising a genomic biomarker from thespecific patient’s molecular report information, to generate a first setof response data; (viii) using the first set of response data togenerate a response; and (ix) providing the response.
 2. The method ofclaim 1, wherein at least one of the qualifying parameter valuesincludes a patient identity, a patient’s disease state, a geneticmutation, or a procedure type.
 3. The method of claim 1, furtherincluding the step of storing a general knowledge database in the systemdatabase that includes non-patient specific data about specific topics,wherein the first data operation is associated with the specificpatient’s molecular report information and the second data operation isassociated with the general knowledge database.
 4. The method of claim3, wherein the second data operation associated with the generalknowledge database is executed first to generate second data operationresults, the second data operation results are used to define the firstdata operation and the first data operation associated with the specificpatient’s molecular report information is executed second to generatethe first set of response data.
 5. The method of claim 3, wherein thefirst data operation associated with the specific patient’s molecularreport information is executed first to generate first data operationresults, the first data operation results are used to define the seconddata operation and the second data operation associated with the generalknowledge database is executed second to generate the first set ofresponse data.
 6. The method of claim 1, wherein the step of identifyingat least one intent includes determining that the query is associatedwith the specific patient, accessing the specific patient’s molecularreport information, determining the specific patient’s cancer state fromthe molecular report information and then selecting an intent from apool of cancer state related intents.
 7. The method of claim 6, furtherincluding the step of storing a general knowledge database in the systemdatabase that includes non-patient specific data about specific topics,the method further including the steps of, upon determining that thequery is not associated with any specific patient, selecting an intentthat is associated with the general knowledge database.
 8. The method ofclaim 1, wherein the collaboration device includes at least one visualindicator, the processor linked to the visual indicator and controllableto change at least some aspect of the appearance of the visual indicatorto indicate different states of the collaboration device.
 9. The methodof claim 1, wherein the query is an audible query, and wherein theprocessor is programmed to monitor microphone input to identify a “wakeup” phrase, the processor monitoring for the audible query after thewake up phrase is detected.
 10. The method of claim 1, wherein a seriesof queries are received, the at least one of the identified first dataoperations including identifying a subset of data that is usable withsubsequent queries to identify intents associated with the subsequentqueries.
 11. The method of claim 1, further including the steps of,based on at least one received query and related data in a systemdatabase, identifying at least one activity that a collaboration deviceuser may want to perform and initiating the at least one activity. 12.The method of claim 11, wherein the step of initiating the at least oneactivity includes generating a second response and providing the secondresponse to the user seeking verification that the at least one activityshould be performed and monitoring for an affirmative response and, uponreceiving an affirmative response, initiating the at least one activity.13. The method of claim 11, wherein the at least one activity includesperiodically capturing health information from electronic health recordsincluded in the system database.
 14. The method of claim 11, wherein theat least one activity includes checking status of an existing clinicalor lab order, or ordering a new clinical or lab order.
 15. The method ofclaim 11, wherein the step of initiating the at least one activityincludes automatically initiating the at least one activity without anyinitiating input from the user.
 16. The method of claim 1, furtherincluding storing and maintaining a general cancer knowledge database,persistently updating the specific patient’s molecular reportinformation, automatically identifying at least one intent andassociated data operation related to the general cancer knowledgedatabase based on the specific patient’s molecular report information,persistently executing the associated data operation on the generalcancer knowledge database to generate a new set of response data notpreviously generated and, upon generating a new set of response data,using the new set of response data to generate another response andproviding the another response.
 17. The method of claim 1 also for usewith an electronic health records system that maintains health recordsassociated with a plurality of patients including the specific patient,the method further including identifying a third data operationassociated with the at least one intent and executing the third dataoperation on the specific patient’s health record to generate additionalresponse data.
 18. The method of claim 17 wherein the step of using thefirst set of response data to generate a response includes using thefirst set of response data and the additional response data to generatethe response.
 19. A system, comprising: a computer including aprocessing device, the processing device configured to:(i) storemolecular report information for a plurality of patients in a systemdatabase; (ii) receive a query answerable in part by reference to agenomic biomarker from the user; (iii) identify at least one intentassociated with the query, the at least one intent including at leastone qualifying entity, the at least one intent identified by a machinelearning module recognizing the query as sufficiently corresponding toan intent phrase provided to or generated by the machine learningmodule; (iv) identify at least one parameter value associated with theentity in the query corresponding to the at least one qualifying entity;(v) identify at least one first data operation associated with the atleast one intent, the at least one first data operation associated withthe at least one qualifying entity; (vi) identify at least one seconddata operation associated with the genomic biomarker; (vii) execute (a)at least one of the identified at least one first data operations and(b) at least one of the identified at least one second data operationson a first set of data in the system database, the first set of datacomprising a genomic biomarker from the specific patient’s molecularreport information, to generate a first set of response data; (viii) usethe first set of response data to generate a response; and (ix) providethe response.
 20. A non-transitory computer readable medium, comprisinginstructions for causing a computer to: (i) store molecular reportinformation for a plurality of patients in a system database; (ii)receive a query answerable in part by reference to a genomic biomarkerfrom the user; (iii) identify at least one intent associated with thequery, the at least one intent including at least one qualifying entity,the at least one intent identified by a machine learning modulerecognizing the query as sufficiently corresponding to an intent phraseprovided to or generated by the machine learning module; (iv) identifyat least one parameter value associated with the entity in the querycorresponding to the at least one qualifying entity; (v) identify atleast one first data operation associated with the at least one intent,the at least one first data operation associated with the at least onequalifying entity; (vi) identify at least one second data operationassociated with the genomic biomarker; (vii) execute (a) at least one ofthe identified at least one first data operations and (b) at least oneof the identified at least one second data operations on a first set ofdata in the system database, the first set of data comprising a genomicbiomarker from the specific patient’s molecular report information, togenerate a first set of response data; (viii) use the first set ofresponse data to generate a response; and (ix) provide the response.